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ABSTRACT 

The second volume of the Commission on Tests' report 
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for Providing Information about Colleges to Applicants; Education and 
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a Machine Adirit an Applicant to Continuing Education? See also ED 039 
396 and TM 0CC 269. (CK) 
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At their meeting on September 24, 1970, the Trustees of the 
College Entrance Examination Board received the final report 3 

of the Commission on Tests and authorized its publication and | 

distribution to the member institutions and to the public at large. \ 

To encourage the fullest consideration and discussion of the | 

report within Board councils, the Trustees also asked their Com- \ 

mittee on Planning and Development to review the entire docu- \ 

ment and make such recommendations as may be appropriate. \ 
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Preface 



This volume contains the primary source documents for Report 
of the Commission on Tests: I. Righting the Balance ; 1 taken to- 
gether these two volumes are the report of the Commission on 
Tests to the College Entrance Examination Board. The preface 
to Righting the Balance gives an accounting of the Commission’s 
procedures since 1967 when its members were appointed by 
the Board for a three-year tenure and charged with undertaking 
a thorough and critical review of the College Board’s testing 
functions in American education, with considering possibilities 
for fundamental changes in the present College Board tests and 
their use, and with making recommendations based on their 
conclusions. 

The first chapter of Righting the Balance recounts briefly the 
history and development of the College Board and its services 
so that readers can understand the context in which the Com- 
mission’s work is received. The remainder of that volume ex- 
plains the consensus that emerged around or is implicit in the 
materials that have been prepared by individual members of the 
Commission and are contained in this volume of the Commis- 
ion’s report. Thus while Righting the Balance is largely self- 
contained, the volume in hand can best be read in conjunction 
with it. 

Midway in the Commission’s tenure individual members of 
the Commission were invited by its editorial committee to sub- 
mit “briefs” to the full Commission for its consideration. These 
briefs were to be short papers arguing for recommendations 
addressed to issues that the Commission had previously con- 
fronted or to other issues of the members’ own choosing. The 
response to this request was gratifying; within a year or so the 
Commission had received from 12 members 14 briefs which 
argued for some 46 proposed recommendations (they are listed 
on pages 187-194). 

1. Righting (he Balance may be obtained from College Entrance Examination 
Board, Publications Order Office, Box 592, Princeton, New Jersey 08540. 
S2 per copy. 
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These briefs, which are collected here in their final form, 
were discussed in meetings of the full Commission and subse- 
quently revised as far as their authors chose to revise them in 
order to gain increased support for their proposed recommenda- 
tions. After discussion and revision were complete, members of 
the Commission were invited to “vote” on their fellow mem- 
bers’ proposed recommendations by having their names listed as 
4 ‘endorsing,” “not endorsing,” or “abstaining” in addenda that 
accompany the briefs. These addenda also contain whatever 
comment, reservation, or dissent members submitted to accom- 
pany their “votes.” 

Most of these individually proposed recommendations re- 
ceived at least qualified endorsement from most members of the 
Commission. In the end, however, the individual members, act- 
ing as a commission, decided to forward these individually pro- 
posed recommendations to the College Board as “suggestions” 
for the Board’s consideration. The status of “recommendation” 
was reserved for a single general proposition, presented in 
Righting the Balance : that the College Board adopt the full range 
of potential entrants into programs of postsecondary education 
as a clientele that is as valued as the Board’s institutional clientele 
and thus is due as large a share of service. 

That recommendation and other conclusions of the Com- 
mission that are at least implicit in the materials in this volume 
are supported in Righting the Balance. Among those conclusions 
is a conviction that the College Board should continue, rather 
than abandon, its testing functions in American education, and 
that the Board’s testing programs and associated services could 
and should serve three broad functions in the interests of both 
students and institutions: 

» — a “distributive” function by contributing to comprehensive 

and sensitive descriptions of students, of colleges and their 
programs, and of the potential relationships between the two 
as both students and colleges engage in a process of recip- 
rocal choice; 
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— a “crcdcntialing” function by certifying demonstrable educa- 
tional attainment whether acquired by attendance in school 
or college or not; 

— an “educative” function by instructing students both in sub- 
ject-matter areas and in the skills and methods of making 
decisions and choosing. 

The Commission recommended that in seeking to serve these 
functions the College Board should be as responsive to the in- 
terests of students as it is to the interests of institutions. This 
volume of the Commission’s report contains its suggestions— 
generated by its members— about the implications, in a system 
of mass postsccondary education, of the Board’s acting for both 
its traditional institutional clientele and an equally valued stu- 
dent clientele. 

The Commission’s suggestions— its members’ individual 
proposals— vary in their nature and arc not equally acceptable to 
all its members. Some urge developments or adjustments in the 
Board’s current tests and associated services. Others foresee en- 
tirely new tests or services, and still others propose changes in 
the way in which the Board is governed or its testing programs 
organized, in order to provide a more facilitativc context for 
balanced service to the College Board’s multiplicity of clienteles. 
These proposals also vary enormously m she scale of resources, in 
the preliminary amount of research and development, and in the 
time required to implement them. No attempt has been made 
to rank them in terms of priority since they have varying de- 
grees of endorsement and since ranking them would involve 
judgments about administrative, managerial, and technical 
matters that the Commission docs not feel competent to make. 

Some readers will no doubt see implications other than those 
expressed here of the Board’s trying, in the public interest, to 
serve the “distributive,” “credentialing,” and “educative” func- 
tions equally in the interests of students and institutions. The 
Commission hopes that all concerned will advance their own 
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suggestions for consideration, and that the College Board will 
move swiftly to implement whatever the optimum combination 
of all such suggestions seems to be in light of the Board's 
resources. 
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Possible Directions in Which College 

Board Tests of Abilities 

and Learning Capacities Might Be Developed 

John B . Carroll 
The Issue 

Should the College Board’s Scholastic Aptitude Test (sat) be 
continued in its present form? If not, what steps should be taken 
toward the development of a more desirable and acceptable form 
of test? 

Discussion 

Assumptions: 

1. There is, and will continue to be, a symmetric 1 relation be- 
tween the student seeking access to higher education and the in- 
stitutions seeking students able to profit from their educational 
programs. On the one hand, the student has the right to demon- 
strate his potentialities for profiting from such programs, and on 
the other hand the institution has the right to require such dem- 
onstration, in view of the fact that its resources are not unlim- 
ited. 

2. Institutions will continue to vary widely in the nature and 
intellectual level of their educational programs. Some institu- 
tions will emphasize liberal arts, humanities, and sciences; others 
will emphasize engineering and technology, while still others 
will emphasize vocational areas such as business, computer pro- 
graming, home economics, industrial arts, and so forth. Some in- 
stitutions will have programs that will be completed satisfactorily 
only by individuals with a rather high level of intellectual ability 
and rapid learning capacity; other institutions will have pro- 
grams that are geared to individuals of lesser intellectual ability 



i. The notion of symmetry impliea here is not the same as the notion implied 
in some other parts of the report of f 'ie Commission on Tests, namely that the 
student should have as much r^ht learn about colleges he might choose as 
colleges have to learn about him. These two notions of symmetry arc indepen- 
dent and unrelated; neither excludes the other, nor are they in conflict in any 
way. 
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and slower learning rates. Even in institutions that themselves 
contain a variety of programs designed for different types of stu- 
dents, there will be need for information about the varying po- 
tentialities of students for these programs. 

3. Students* performances on tests can afford one good and 
valid source of information on potentialities. While test per- 
formances arc not the only source of such information, students 
will continue to want to use their test performances as evidence 
of potentiality, and institutions will continue to accept, and in 
most cases require, such evidence along with other kinds of evi- 
dence. (It would be interesting to investigate the consequences 
of making the taking of tests optional for students in certain 
cases.) 

4. It would be feasible from the standpoints both of scientific 
research and of administrative considerations to develop a series 
of tests that would better indicate differential potentiality for 
educational programs, and that would have a better distributive 
effect, than the present sat. (The present sat tends to have a 
negative distributive effect in that it is geared more for selecting 
students for the better institutions than for indicating types of 
institutional programs from which the student is most likely to 
profit.) 

The Present sat 

The sat is in truth a test of developed abilities, depending both on 
general intellective capacities to learn and on an accumulation 
of knowledge and skills acquired through education in, and ex- 
perience with, the verbal and mathematical aspects of this na- 
tion’s culture. The SAT-Verbal sections emphasize verbal skills — 
knowledge of English vocabulary, word usage, and reading 
comprehension— while the SAT-Mathematical sections involve 
the ability to deal with quantitative and geometric-spatial con- 
cepts. Both sections also measure reasoning and inferential abil- 
ity, SAT-Verbal in the verbal domain and SAT-Mathematical in 



the quantitative domain. Typically, the correlation between sat- 
Verbal and SAT-Mathematical scores has been about 65 in the 
population taking the sat; this reflects not only the common in- 
volvement of the reasoning element, but also the fact that both 
sections call for ability to comprehend printed language and abil- 
ity to answer questions at the required pace. (It cannot be denied 
that there remains a “speed” element in the test, at least for a 
substantial proportion of the population, even though its impor- 
tance has been reduced in recent years!) - 

From the standpoint of factor-analytic studies, therefore, the 
sat reflects at least four “factors” of ability: (1) verbal knowl- 
edge— that is, knowledge of vocabulary and ability to under- 
stand language; (2) ability to deal with quantitative and spatial 
concepts; (3) ability to reason with concepts either in verbal or 
quantitative terms; and (4) speed in test-making. The two scores 
that are reported reflect different proportions of these abilities. 

There is logic and massive evidence to support the proposi- 
tion that the skills measured by the sat in its present form are re- 
quired at fairly high levels if the typical student is to succeed in, 
and profit from, his educational experience in colleges and uni- 
versities, particularly the ones that are more selective and that 
emphasize liberal arts and sciences in their programs. Learning 
at the higher-education level involves acquiring information, 
skills, and principles from books, lectures, oral discussions, and 
other educational media that carry verbal discourse. Some de- 
gree of verbal skill is required even in the less selective institu- 
tions, and even in the “free universities” organized by students 
at some campuses. (In fact, one has the impression that “free uni- 
versities” are for the most part organized by and for students 
with high degrees of verbal skill. Of course, the students in these 
“free universities” would likely abhor the idea of setting up ad- 
missions requirements for them, least of all requirements based 
on admissions test performance.) It can be said, then, that verbal 
and reasoning skills are highly relevant to success in higher edu- 
cation, in nearly every type of educational program that exists or 
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that might be conceived. Quantitative reasoning ability is also 
relevant, but perhaps to a lesser extent; it plays a particularly im- 
portant role in scientific and technological programs. 

There is some uncertainty about the degree to which these 
skills can be modified. The bulk of the evidence available sug- 
gests that verbal, mathematical, and reasoning skills as measured 
by the SAT are acquired relatively slowly throughout the indivi- 
dual’s preuniversity years, and that they cannot be quickly or 
easily modified, even when the student has high motivation to 
do so. In the case of students who have had average educational 
experiences but who make low scores on the SAT, one may esti- 
mate that even a year of rather intensive remedial instruction in 
verbal and reasoning skills would not generally suffice to make a 
dramatic improvement in test performance or in potentiality for 
success in higher education. Several studies have led to the same 
rather discouraging conclusion for groups of “educationally 
disadvantaged” students. However, a number of colleges and 
universities have had some success with admitting low-scoring 
educationally disadvantaged students and giving them the special 
attention they need to succeed. One must note that the students 
admitted to such programs have generally been hand-picked on 
the basis of various kinds of information other than sat test 
scores. It would seem that at least some of these students had po- 
tentials that were not adequately measured by the sat. 

If the assumptions underlying this brief are accepted, the 
above discussion leads to the conclusion that some aspects of the 
present sat must be retained in the Board’s program, but that 
the sat must be modified and expanded to make it a more effec- 
tive and multipurpose instrument. For the great majority of stu- 
dents with “average” educational experiences, it must be 
shaped so that it will more efficiendy and differentially measure 
the skills that are most relevant to success in higher education. 
At the same time, it must provide a better means of measuring 
the educational potential of candidates who have for one reason 
or another been educationally disadvantaged. 



Possible Changes in the sat 

An SAT can be envisioned that would look more like a test bat- 
tery consisting of a number of component parts than the omni- 
bus test that it now is. Even allowing for a half-hour experi- 
mental section, it should be possible in a three-hour testing 
period to gamer more information about the student than is 
provided by the present test with its two scores, SAT-Verbal and 
SAT-Mathematical. The score report would contain four or five 
separate scores, enabling the student to learn more about his 
abilities and prospects and permitting the admissions officer to 
get a more detailed picture of the candidate. Whereas the present 
SAT tells the student about his chances of success in the more se- 
lective institutions, a new sat would also inform him about his 
chances of success in a wider variety of types of institutions and 
programs. 

The following changes can be suggested: 

1. To yield more discriminating scores, the verbal knowledge 
and verbal reasoning elements of the sat should be more clearly 
separated by appropriate item-analysis and test-construction pro- 
cedures. The verbal knowledge elements should have their rea- 
soning components reduced in importance, yielding a score that 
would mainly give evidence of how well the candidate can com- 
prehend language. The reasoning elements, on the other hand, 
should have the verbal knowledge components reduced in 
importance, yielding a score that would indicate how well the 
student can reason with materials he already understands. Pre- 
sumably, the former score would be largely a function of the in- 
dividual^ education and general reading experience, while the 
latter score would be less influenced by these factors and be more 
predictive of success for individuals with educational disadvan- 
tages. 

2. Consideration should be given to the desirability of reduc- 
ing in importance the “mathematical” elements in die sat. If an 
SAT-Mathematical score is reported at all, it could be based on a 
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shorter test (rather than the two sections now used). Reducing 
the mathematical sections in importance would not result in any 
great loss in validity for the majority of students, even for those 
students who might enroll in courses requiring quantitative rea- 
soning at the more selective institutions. (The SAT-Verbal score is 
apparently almost as predictive as the SAT-Mathematical score 
for such courses.) Institutions whose programs are predomi- 
nantly scientific and technological would be advised to rely, for 
admissions information, on mathematics achievement tests. The 
present College Board Mathematics Achievement Tests in turn 
might be expanded to include some materials of the type now 
found in the SAT-Mathematical sections. At the same time, there 
is some evidence that many educationally disadvantaged stu- 
dents are able to show much better performance on the sat- 
Mathematical sections than on the SAT-Verbal sections: to the ex- 
tent that such performance is a relevant indication of unrealized 
potential, weight is added to the argument that some features of 
the SAT-Mathematical sections should be retained in a revised 
sat battery . 

3. Some parts of the sat might be administered essentially as 
listening comprehension tests— for example the verbal reasoning 
sections— with minimal involvement of the reading element, in 
order to tap the potentials of those who have good basic verbal 
reasoning ability but have special difficulty with reading. Such 
a test might in fact be equally effective and valid even for those 
with good reading ability. Alternatively, the verbal knowledge 
and verbal reasoning sections would be retained in printed form, 
but a separate listening comprehension section would be in- 
cluded. 

4. An effort should be made to develop a section of the sat 
that would better measure various learning capacities apart from 
developed abilities. Over the years, there have been indications 
in research literature that such learning capacities can be identi- 
fied and measured, but there has never been a concentrated, 
major effort to develop the necessary instruments. Such tests 
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might better reveal the educational potential of students who for 
one reason or another have not had the advantages of a good 
education. Exactly what forms such tests might take would have 
to be determined by research. They might, for example, consist 
of work-sample learning tasks ranging in complexity from 
simple serial learning to more involved tasks calling for the 
learning of new verbal concepts, rules, and principles. In any 
event, every effort should be made to reduce the dependence of 
such tests on educational opportunity, vocabulary knowledge, 
reading ability, or the like. 

5. Present methods of validating SAT items militate in favor 
of items that measure developed abilities specifically useful in the 
more selective institutions and against items or tasks that would 
measure learning potential. Insofar as the criterion is overall col- 
lege success in typical liberal arts institutions, it favors items or 
tests that measure the extent to which students have already de- 
veloped their abilities to the point of being comfortable in a 
scholastic environment that makes demands on previously de- 
veloped knowledge. While this criterion may be useful for cer- 
tain sections of the proposed new sat, for example the verbal 
knowledge section, a series of different criteria should be used 
for the other sections. Success in junior colleges, vocationally 
oriented institutions, and other different types of institu- 
tions would be used as criteria. The criterion for the “learn- 
ing ability” section would be one that measures the extent 
to which a student can develop his abilities in a college environ- 
ment. 

Recommendations 

It is recommended that the College Entrance Examination Board: 

1. Support research to investigate the feasibility of recasting 
the SAT along the following lines: 

a. provision of a section or sections more clearly emphasizing 
developed verbal skills relating to language comprehension and 
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learning from language, with the reasoning component de- 
em phasizcd; 

b. provision of a section or sections more clearly emphasize 
ing ability in reasoning and inference that does not depend 
greatly on verbal skills; 

c. de-emphasis of the quantitative sections of the tests, with 
provision for testing of developed ability in this area in specific 
achievement tests apart from the sat; 

d. administration of some parts of the sat in an auditory 
mode, to reduce dependence of those parts on reading ability; 

c. provision of a section or sections that would measure scho- 
lastic learning capacities or potentials as much as possible apart 
from developed abilities; 

f. validation of sat materials not only against overall absolute 
success in higher education (in various types of programs), but 
also against gains made in developed abilities. 

2. Introduce on a gradual basis, if the above types of tests 
prove feasible and acceptable, a new and possibly renamed sat, 
with appropriate dissemination of information about the test 
both to educational institutions and to candidate populations. 



Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
Possible Directions in Which 
College Board Tests of Abilities and 
Learning Capacities Might Be Developed 



It is recommended that the College Entrance Examination Board: 

I. Support research to investigate the feasibility of recasting the 
SAT along the following lines: 

a. provision of a section or sections more clearly emphasizing 



developed verbal skills relating to language comprehension and 
learning from language, with the reasoning component de-em- 
phasized; 

b. provision of a section or sections more clearly emphasizing 
ability in reasoning and inference that docs not depend greatly 
on verbal skills; 

c. de-emphasis of the quantitative sections of the tests, with 
provision for testing of developed ability in this area in specific 
achievement tests apart from the sat; 

d. administration of some parts of the sat in an auditory 
mode, to reduce dependence of those parts on reading ability; 

c. provision of a section or sections that would measure scho- 
lastic learning capacities or potentials as much as possible apart 
from developed abilities; 

f. validation of sat materials not only against overall absolute 
success in higher education (in various types of programs), but 
also against gains made in developed abilities. 

Endorsed by lg members: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Hechingcr, Herscy, Holtzman, 
Hoy, Marland, Pace, Pearson, Prentice, Schlossberg, Shirley, 
Thresher, Tiedeman. 

Hcrsey: “The third suggestion (c) under Recommendation 
I, even with its proviso of achievement testing of developed 
quantitative abilities, will be perceived as working a hardship on 
disadvantaged students. This leads to a question: Since it is rec- 
ommended that the ‘developed 1 abilities be distinguished from, 
yet be kept in the same test with, reasoning skills and aural re- 
sponses and learning capacities, would it not be desirable to in- 
clude all of these and achievement tests in one overall ‘test, 1 or 
battery of tests, yielding a series of differentiated scores? This 
would help to take the curse off reducing the quantitative com- 
ponent of the ‘aptitude* test and shifting whatever emphasis it 
may provide into an entirely different category of test.** 

Marland: “This brief should declare straightforwardly the 
need for adaptation to the young people of minority populations 
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and others living in handicapped environments. ” 

Prentice: “How important is the tendency for educationally 
disadvantaged students to show better performance on the 
mathematical sections of the SAT than on the verbal? Does Car- 
roll^ proposal do justice to that aspect of the problem?” 

Schlossberg: “My endorsement carries a basic reservation.” 
Shirley: “On the whole I am totally sympathetic with the 
objectives and the direction proposed by Carroll in both Rec- 
ommendations I and 2. There are, however, a few comments 
and perhaps slight disagreements. 

“a. and b. This would be a good move, if it can be accom- 
plished. Much of what Carroll says in his discussion on pages 3, 
4, and 5 seems to indicate that this is difficult, since reasoning 
may depend largely on verbal ability. Certainly the process of 
verbalization calls for reasoning, and I sometimes feel that we 
think (in logical, reasoning terms) only when we move to verbal- 
ization. How much logical thought is a byproduct of language 
we do not know; experiments with other animals seem to indi- 
cate some connection. But if separation there can be, so let it be. 

“c. Carroll’s de-emphasis of the quantitative portion of the 
SAT seems again to be related to the importance of verbalization. 
In the two universities with which I have been connected, we 
have not found the high correlation between SAT-V and the sci- 
ence and engineering programs that Carroll suggests exists. We 
have found considerably higher prediction success with sat-M. 
True, the Mathematics Achievement Tests help but little, even 
for ability-level sectioning. 

“d. The question about the culture bias of the written lan- 
guage is nothing compared with the culture loading of the 
spoken. What spoken language? Must this oral portion of the 
examination be localized to the dialect of the particular commu- 
nity or the economic level and racial background of the individ- 
ual? 

“c. This was originally the purpose of the Scholastic Aptitude 
Test; these tests have come to test content achievement. We have 
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been told that this is all that can be done, since this is the thing 
that leads to success in college. It has long been my contention 
that this really meant that validation was against only one kind 
of higher education, represented by the New England liberal 
arts institution, which had little relevance to the complex public 
university. Personally, I don’t care whether these tests measure 
one or the other, so long as they are helpful in placing students 
into situations in which they can be successful. 

“f. A very important point, on which ets should conduct re- 
search immediately. I have faith in this suggestion.” 

Thresher: “It would be captious not to endorse these mild, 
ameliorative proposals, all well within the established psycho- 
metric tradition. My only fear is that, by accepting this kind of 
list we delude ourselves into thinking the job is done. Accept- 
ance must not preclude active experimentation with some of the 
more imaginative suggestions we have discussed, nor prevent us 
from working away from a ‘package* sat with set parts and into 
a wider flexibility of combinations. It should not imply uncrit- 
ical acceptance of conventional test criteria such as college 1 suc- 
cess.* 

“If e. turns out to be possible, I shall be pleased and surprised. 

“As to f., I see a certain circularity in using one test to serve 
as criterion for another.** 

2. Introduce on a gradual basis, if the above types of tests prove 
feasible and acceptable, a new and possibly renamed SAT, with 
appropriate dissemination of information about the test both to 
educational institutions and to candidate populations. 

Endorsed by 19 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Hechinger, Hersey, Holtzman, 
Hoy, Marland, Pace, Pearson, Prentice, Schlossberg, Shirley, 
Thresher, Tiedeman. 

Shirley: ‘This certainly should be done. The College Board 
has been concerned that ets does not perform more such experi- 
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mentation and research on these bread-and-butter programs of 
the two institutions. If this suggestion were made to the Board’s 
trustees, I am certain they would be overjoyed. Let the Com- 
mission make this strong recommendation— both to the Board 
and to ets.” 

General Comments 

Hersey: “While I am very much in favor of the body of this 
brief, I still find myself uncomfortable with the first paragraph. 
Despite the disclaimer in the footnote, I feel that the two notions 
of symmetry are related, and antithetically so. The symmetry 
that the Commission has discussed as new and desirable seems to 
me to emphasize the reciprocal benefits of a process of choice in 
which colleges learn about applicants, applicants learn about col- 
leges, and both learn about themselves; the symmetry in Car- 
roll’s brief seems to me to hark back to the status quo— a one- 
sided process of selection by a college of the students it wants. 
While the Carroll formulation does rather accurately describe 
things as they are, it tends to blunt the other more potential idea 
of symmetry. Furthermore, it does not seem to me to be essential 
to the logical argument of the brief, which might very well be- 
gin with what is now the second paragraph.” 

Tiedeman: “I disagree with the first assumption, at the begin- 
ning of the brief, despite the footnote. The right attributed to 
the student is contingent on the right exercised by the institu- 
tion. Therefore it is not a student right, it is a student necessity 
because of institutional action. This is a symmetry with only one 
degree of freedom. It is the symmetry on which College Board 
work is now predicated. I would like to see such predication 
eliminated.” 
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Redundant Testing 

John B. Carroll 



The Issue 

Can the amount of multiple test-taking in college admissions 
testing be reduced while preserving the validity and meaning of 
the test scores and also preserving the present diversity of tests 
and the healthy competition among testing organizations and 
agencies? 

Discussion 

While exact figures are not available, there is considerable evi- 
dence that many students throughout the nation are now being 
required to take a number of more or less similar tests for college 
admission or scholarships or both when they apply to different 
colleges. Students and parents complain about the apparently 
needless time spent in taking tests beyond a reasonable number, 
and school administrators object to the undue interference in 
school programs caused by a multiplicity of testing programs. 
Besides the College Entrance Examination Board program, there 
are the American College Testing Program and the National 
Merit Scholarship Qualifying Program, as well as a number of 
special testing programs administered by private, state, and na- 
tional agencies. 

As noted above, the tests are generally similar to one another; 
all measure, with minor variations, the same sorts of abilities as 
are measured by the College Board’s Scholastic Aptitude Test 
(sat). One may presume, on the basis of some available evidence 
and by inference from general findings in the field of testing, 
that the intercor relations among the various tests would be 
found to be quite substantial and in some cases almost as high as 
would be permitted by the test reliabilities. The tests have, how- 
ever, been normed on different populations; this is one reason 
why it is difficult if not impossible to set up accurate equivalen- 
cies among scores from different tests. And even if equating ex- 
periments were conducted, there arc many technical sources of 
error in the establishment of equivalencies. These technical mat- 
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ters have been discussed in detail in a paper that W. H. AngofF 
presented at the 1962 Educational Testing Service Invitational 
Conference on Testing Problems. 1 

The problem of redundant testing is also beset with delicate 
“political** problems. Many of the college admissions and schol- 
arship tests are offered by private organizations (both profit and 
nonprofit) that are understandably occupied with protecting 
their interests. They operate in separate spheres and tend to be 
unconcerned about the overlap of those spheres as far as stu- 
dents taking the tests are involved. As a result, one group of col- 
leges requires a certain admissions test (for example, the sat), 
while another group (rarely overlapping with the first) requires 
another admissions test (for example, the American College Test- 
ing Program tests). Many students feel obligated to take both 
tests, to improve their chances of admission. On the other hand, 
it can be argued that the healthy competition among a number 
of different testing agencies and programs has resulted in overall 
improvement in testing procedures. It would be unfortunate to 
lose the benefits of this competition. 

The solution to the problem of redundant testing, therefore, 
is partly technical and partly political. 

On the technical side, it would be necessary to set up exten- 
sive equivalency studies to obtain more evidence about the simi- 
larity of the tests and the possibility of establishing tables of 
equivalent scores. One technique for establishing equivalencies 
has not been adequately investigated in this context: the tech- 
nique of using a third variable. That third variable should be, it 
would seem, a measure of the criterion, that is a measure of suc- 
cess in college. Under this method, briefly discussed by AngofF, 
scores on two tests would be regarded as equivalent if they pre- 
dicted equal values on the criterion. While subject to certain er- 
rors, this method is not subject to the same errors as the usual 



I. “Can Useful General Purpose Equivalency Tables Be Prepared for Different 
College Admissions Tests?” in Anne Anastasi, cd., Testing Problems in Perspec- 
tive. Washington, D.C.: American Council on Education, 1966, pp. 251-264. 



parallel-form method, and its errors can be tolerated in the col- 
lege admissions situation since they are precisely the errors that 
arise in prediction anyway. In fact, the method would reveal the 
nonexistence of scores, on a relatively invalid test, that would be 
equivalent to certain scores on a relatively more valid test; thus, 
tests would be put in competition for validity. Use of this me- 
thod in research on test equivalence would not even require that 
students take more than a single test as long as all students in- 
volved in a given comparison have been measured on a common 
criterion variable. 

There would be problems, of course, in selecting such crite- 
rion variables. In other briefs of the Commission on Tests it has 
been suggested that multiple criteria be used, and if tests such as 
the sat are broken dov/n into several components the problem 
of establishing equivalencies becomes even more exacerbated. 
Nevertheless, it should be possible to establish reasonably ade- 
quate equivalencies among different testing programs as such, 
even if some compromise has to be made with technical issues. 

On the political side, measures would have to be taken to (i) 
get the various testing agencies to agree to cooperate in equiva- 
lency research, and (2) get the colleges to agree to loosen their 
admissions test requirements in such a way as to accept testing 
information from either of several testing programs. It is true 
that such measures might lead to a reduced volume of testing 
and resultant income for the testing agencies, but in the public 
interest such reductions should be tolerated and even encour- 
aged. 

Recommendations 

It is recommended that the College Entrance Examination Board: 

1. Propose to an organization such as the American Council 
on Education that it create a Council on College Admissions and 
Scholarship Testing that would invite the various testing organi- 
zations and agencies to join for the purpose of cooperating in re- 
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search that would lead to mutual agreements among themselves 
and the higher-education institutions concerned, whereby col- 
leges would accept scores on different tests as equivalent for ad- 
missions purposes. 

2 . Propose that the research and development to be spon- 
sored by this council be based on a method of establishing equiv- 
alencies in terms of equivalent predictions of criterion variables. 



Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
Redundant Testing 

It is recommended that the College Entrance Examination Board: 

i. Propose to an organization such as the American Council on 
Education that it create a Council on College Admissions and 
Scholarship Testing that would invite the various testing organi- 
zations and agencies to join for the purpose of cooperating in re- 
search that would lead to mutual agreements among themselves 
and the higher education institutions concerned, whereby col- 
leges would accept scores on different tests as equivalent for ad- 
missions purposes. 

Endorsed by 17 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Gordon, Hechinger, Hcrsey, Holtzman, Hoy, Marland, 
Pearson, Prentice, Schlossberg, Shirley, Thresher, Ticdcman. 

Coleman: “I believe there arc serious dangers of reduction of 
competition between testing agencies if the emphasis in such a 
Council as proposed is on equivalencies between tests rather than 
on acceptance of different tests by the colleges. Equivalence is 
necessary in order to compare students on a single dimension of 
college success. But if the tests from a single agency themselves 
become more differentiated (as has been proposed in another 
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brief by Carroll), then the use of the tests by admissions officers 
must itself undergo change, since these officers will not have a 
single score to use for comparison of students. What should 
emerge is the use of regression equations to predict performance 
in that college, and the various test scores would be variables in 
the prediction equation, with different weights in different col- 
leges. With this use of the test scores (as is implied by Carroll’s 
other brief), it becomes simple and straightforward for a college 
to use more than one testing program. One regression equation 
would be used for the test scores from one agency; another 
would be used for the test scores from a second agency; and still 
a third could incorporate test scores from both. Such an approach 
would also have the virtue of stimulating testing agencies toward 
diversity in testing, in order to provide tests that add to die exist- 
ing predictive ability of current tests. In contrast, the creation of 
direct equivalencies tends to reinforce the existing single-test ap- 
proach and to create more pressure for uniformity.” 

Not endorsed by 2 members: Friedenberg, Pace. 

2. Propose that research and development to be sponsored by this 
Council be based on a method of establishing equivalencies in 
terms of equivalent predictions of criterion variables. 

Endorsed by 16 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Gordon, Hechingcr, Hersey, Holtzman, Hoy, Marland, 
Pearson, Prentice, Schlossberg, Shirley, Tiedenian. 

Foster: “Where possible.” 

Not endorsed by 3 members: Friedenberg, Pace, Thresher. 

Pace: “Considering the likelihood that the ‘criteria’ arc 
changing, one runs the risk of equating tests on ‘status quo* cri- 
teria or on criteria that will sooner or later be obsolete. The 
method is one that may tend to inhibit potentially desirable 
changes in the values and performances rewarded by higher edu- 
cation. At this point in time, I think it preferable to let compet- 



mg test agencies each work toward adapting their tests to new 
criteria (as Carroll proposes to revise the sat to serve broader 
purposes) rather than to equate present tests to some common 
criterion. I agree with Carroll that redundant testing is an annoy- 
ance for some students and high school administrators; but I 
think that this annoyance can be tolerated.” 

General Comments 

Holtzman : “Nothing wrong with the proposal, but I’m skeptical 
of its working.” 

Shirley: “An excellent proposal and one that, in the national 
interest, I hope the Commission can help to implement as rapidly 
as possible. Such a recommendation should also be directed to 
the American College Testing Program, as well as to the College 
Board, to try to get cooperation rather than pure competition 
between them.” 

Thresher: “Obviously desirable as a practical measure within 
the limits of the current psychometric tradition, but should not 
be permitted to foreclose continued criticism and questioning of 
that tradition. We should be looking primarily at a wider range 
of experiment.” 
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The Principle of Symmetry 
in College Choice 

James S. Coleman 

Modern society supports at many points a form of relation that 
is relatively new in the history of man. This is the relation be- 
tween an individual and a large organization. Although this kind 
of relation has existed for some time, the pi oliferation of large 
organizations, and the extension of man’s relevant environment 
beyond the borders of his community or neighborhood, have 
only recently made it commonplace. It arises in a man’s employ- 
ment by a large firm, in his purchasing from large producers or 
large retailers, in his dealings with large government. It arises in 
a student’s relation to his college or university. And, what is to 
the point here, it arises first at the time of his choice of college 
and its choice and placement of him. 

The principal problems that arise in this form of relation re- 
sult from the asymmetry in size between the individual and the 
organization, and the asymmetry in power that stems from the 
size disparity. Ordinarily the organization can mobilize far more 
resources to further its interests than can the individual to fur- 
ther his. It has enormous economies of scale, since it is involved 
in similar relations with many individuals. In employment, this 
power differential between the company and the worker has 
been redressed through the construction of countervailing or- 
ganizations, labor unions, in marketing, there is a similar dis- 
crepancy in size and resources between the producer or seller 
and the consumer. Unlike workers, consumers have had little 
success in developing organizations that can balance the power: 
there are few consumer-research organizations to balance the 
market research done for firms, and few organizations that pro- 
vide information to consumers about the reliability of businesses 
to balance the credit bureaus used by retail firms. 

In the choice of a college by a student, and of students by a 
college, this asymmetry in size arises as in employment and mar- 
keting, and it has produced a similar discrepancy in power. The 
colleges have marshaled a number of resources to aid them in 
realizing their aims in selecting and placing students. Colleges 
have established admissions officers, college recruiters, placement 
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officers, and others to determine how best to carry out the ad- 
mission and placement they desire. In addition, they have created 
organizations, most prominently the College Entrance Examina- 
tion Board, to provide them with systematic objective informa- 
tion on applicants that makes it possible to compare individuals. 

The fact that the activities of the college admissions office, 
the placement office, and the external testing agency serve some 
of the interests of the applicant as well as those of the college has 
generally obscured the additional fact that they do so inciden- 
tally, and thus serve his interests less well than those of the col- 
lege. 

This situation can be easily seen by imagining a reversal of 
the power relation. If applicants were organized to have similar 
facilities at their disposal, they would very likely use them to 
provide detailed information about the colleges, to aid in their 
choice among colleges and to provide detailed information 
about various college programs as an aid in their selection of a 
program within college. These activities certainly would be in- 
cidentally in the colleges’ interests, just as the present activities 
of college admissions officers and the College Board testing pro- 
grams are incidentally in the students’ interests, but they would 
no more solve the problems of choice facing the colleges than the 
activities of college admissions officers and the Board now solve 
the problems of choice facing applicants. 

Thus the size asymmetry between the individual applicant 
and the college manifests itself principally in an asymmetry of in- 
formation available to the applicant and to the college, informa- 
tion upon which each makes its choice. The college demands and 
gets specific comparable information from and about applicants: 
high school grades, the information it requests in its admissions 
application form, and often most important of all, scores on the 
College Board’s Scholastic Aptitude Test and Achievement 
Tests, or comparable tests from another agency. The applicant 
has only hearsay, rumors, and whatever information the college 
chooses to exhibit in its catalog as the basis for his selection of 
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college and program of study. If he is fortunate, he has a friend 
attending a college, or he may visit the college and talk to a few 
students there and thus feel that he knows something of the at- 
mosphere. The high frequency of college choices made on the 
basis of such insubstantial and unrepresentative experiences, as 
shown in the few studies made on college choice, is evidence of 
the absence of systematic means by which applicants can assess 
a college. 

It might be argued that since the college has the same goal 
as the individual- -both want him to have a successful college 
career— then the likelihood of a successful match will be as much 
enhanced if the college selection process is merely greatly im- 
proved as it would if the information asymmetry were redressed. 
Apart from the fact that such a solution is possible only for the 
highly selective colleges, and not in the open, nonsclective in- 
stitutions that account for most college students, there arc further 
answers to this argument. First, it appears most reasonable that 
a match between two parties that results from the concurrent 
choices of both parties, based on accurate information, will be 
better than a match based on the choice of one party alone. Sec- 
ond, only if individuals have the opportunity to make choices, 
and to live responsibly with the consequences of their choices, 
can they acquire the independence and sense of responsibility 
that colleges aim to develop. The individual applicant does not 
want to be powerless and passive in the face of the college’s de- 
cision any more than the college wants to be powerless to make 
its choice. Colleges would hardly find it satisfactory to be de- 
prived of the sources of objective and comparative information 
on applicants they now have. Why should individuals find it 
satisfactory to be without objective and comparative information 
that could aid them in their choice? 

The particular aspects of the college-applicant relation, and 
the information control and transmission that occur within it, 
can be expressed by comparing the resources of each, as in the 
following list. 
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Comparison of Applicant and College Resources 



Resource 

Information 
about the 
other. 



Freedom to 
select infor- 
mation to be 
seen by the 
other. 



Search 

mechanisms. 



Applicant 

Information the college 
is willing to reveal in 
its catalog, combined 
with adventitious 
sources of information. 



Each college catalog 
emphasizes what the 
college believes to be 
most attractive. 

Certain information, 
such as attitudes of 
students toward the 
college, morale of 
student body, job- 
placement success of 
graduates, is difficult 
or impossible to obtain. 

Adventitious sources 
of information 
through personal 
acquaintances, 
family experience, 
or geographic 
propinquity. Aid 
from guidance 
counselor in high 
school. 



College 

Specific information 
obtained from each 
applicant: completed 
application form, 
high school grades, 
standardized aptitude 
and achievement test 
results, and independent 
evaluation of applicant 
by letters from persons 
who know him. 

Each applicant must 
provide information that 
allows direct compara- 
bility with other appli- 
cants, based on a 
relatively standardized 
testing program. 



Scholarship testing 
programs to locate 
high-performing stu- 
dents, and sometimes 
recruitment policies 
using lists of scholarship 
winners or near winners. 
When college is 
motivated, as in the case 
of finding good Negro 
applicants or athletes, 
scouts and recruiters 
visit schools and in 
other ways search out 
desired applicants. 
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Resource Applicant 



College 



Information 
about self 
and likeli- 
hood of a 
successful 
match. 



Little objective 
information about 
interests, abilities, 
and knowledge of 
current students 
in the college or the 
college’s special 
program of study. A 
few sources available, 
such as the College 
Board’s College 
Handbook showing 
distributions of test 
scores in some colleges. 



Experience with students 
of various types. In 
most systematic selection 
procedures, knowledge 
of college success of 
students with specific 
characteristics matching 
those of applicant. 



This asymmetry in resources available to individuals and colleges 
by reason of size indicates only that the colleges have realized 
their goals first. Over a period of time, organization in the in- 
terest of applicants can be expected to develop, and with it both 
the resources to provide desired information, and the power to 
induce colleges to release the desired information. There are nu- 
merous indications of this development already: books describ- 
ing colleges, published by commercial firms; suggestions (such 
as one made several years ago by David Riesman) for explicit 
consumer research on colleges, designed to benefit prospective 
applicants; measurement of the social and intellectual character- 
istics of college campuses by questionnaires administered to a 
sample of the student body, such as C. Robert Pace’s College & 
University Environment Scales; the growth of organized student 
activity. 

Redress of the imbalance could come about through the cre- 
ation of explicit, organized groups to further the interests of pro- 
spective college students, or it could arise through the modifica- 
tion of existing institutions. If it is the latter, the disruptive 
period of transition that always arises in the struggle for power 
of a newly organized group will not occur. And if it is the latter, 
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the most obvious agency to incorporate and further these in- 
terests is the College Board itself. To do so will require serious 
modification of the very concept under which the Board op- 
erates, but this concept has already undergone slow evolution 
from the time of the original founding of the Board as a creature 
of the selective Eastern private colleges to the present day when 
the interests of secondary schools and applicants are directly 
taken into account (through the opening of Board membership 
to high schools, publication of college test-score profiles, pilot 
programs to inform disadvantaged youth of college opportu- 
nities, and sponsorships of research such as Pace’s work on the 
characteristics of college environment). 

The recommendations made here are not designed to restrict 
the colleges’ autonomy in their choice— indeed, it is the virtue of 
the system of higher education in America that both colleges and 
students have full freedom of choice, subject to students’ and 
colleges’ interests in each other. Rather, the recommendations 
are designed to lead the College Board to aid applicants’ choices 
as explicitly as it now aids colleges’. 

One major asymmetry is the college’s control of the syste- 
matic and quantitative information about itself that applicants 
receive, and the applicant’s lack of control over the information 
that the college receives about him. Two modifications, em- 
bodied in the first two recommendations of this brief described 
below, are necessary to provide the appropriate symmetry that 
will aid the matching process. 

It is recommended (Recommendation i) that the Board 
systematically gather and publish relevant information about 
colleges, the nature of that information to be determined by a 
working group including representatives of colleges, students, 
and high schools. The information should include such items as 
statistics on test-score distributions of entering freshmen; the 
socioeconomic and geographic origins of entering freshmen; 
the number of students who drop out and transfer in each year 
and their reasons for doing so, including the proportion of cn- 
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tering freshmen who complete college in the normal period; the 
number of each type of degree awarded; the proportion of grad- 
uates going on to professional schools, graduate schools, and 
various types of occupation; standardized statistics on class size, 
number of class hours of teaching per year per faculty member, 
and faculty-student ratio; a measure obtained from student ques- 
tionnaires of the social and intellectual climate and focus of ac- 
tivities in the student body; estimates of the rates of deviant be- 
havior, such as drug use, at the college. It should also include 
more qualitative information of interest to potential students: 
the nature of special programs offered by the institution, the op- 
portunity for seminar and tutorial work, the flexibility of the 
course system and the grading system; and some information 
about the principal emphasis of the institution. 

Such information, with its quantitative parts tabulated to 
facilitate comparison, should be provided in a periodic Board 
publication distributed for the use of potential applicants in high 
schools. 

It is recommended (Recommendation 2) that the Board 
make more symmetric the control of information by colleges 
and by applicants. This can come about through modification of 
the testing program itself. Beyond the standardized testing that 
allows colleges to make direct comparisons of a few character- 
istics of applicants, the applicant should have the opportunity, 
through selection from a very wide variety of special tests rang- 
ing in subject matter from numismatics to algebraic topology, 
to exhibit his special capabilities, developed in or out of high 
school, that may be relevant to the colleges’ interest in him. The 
college, in turn, may choose to take into account or to ignore the 
particular skills exhibited in a given test. Applicants have an 
enormous range of special skills, sometimes developed to a re- 
markable degree as a result of a hobby, which the present testing 
program masks, leaving them unable to show what they are 
best at and failing to give the college a full view of the individ- 
ual capabilities of the student. A secondary beneficial effect of 
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this kind of opportunity would be that it would induce potential 
applicants to pursue independent lines of inquiry that attract 
them during adolescence, unconstrained by the belief that such 
work will only detract from their efforts to be accepted by the 
college of their choice. 

In order to insure realization of the symmetry toward which 
these recommendations aim it is recommended (Recommenda- 
tion 3) that the Board provide colleges and applicants with in- 
formation about the other only if they arc willing to allow in- 
formation about themselves to be made available. In effect, data 
on applicants would be made available to a college only if it al- 
lowed a standardized set of data to be made available to appli- 
cants. (A college at present can insure that an applicant makes 
data on himself available; it may, as many colleges do, refuse to 
consider an applicant for admission unless he provides the full 
set of information about himself requested by the college.) 

These three recommendations aim at increasing the symme- 
try of the Board’s activities and the services it provides. It is clear, 
however, that the appropriate services themselves may change 
over time. Because of the governing structure of the Board, it is 
not possible to make an accurate assessment of the services that 
are most valuable for applicants in making their choice of col- 
lege. Consequently, in order to implement the first three recom- 
mendations appropriately, a change in the governance of the 
Board is necessary. 

The fourth recommendation (Recommendation 4) and the 
most fundamental, is that the College Board modify its govern- 
ing structure, which now includes only representatives of one 
side of the college selection process, to embody a symmetry of 
college and applicant interests. This change can be made in either 
of two ways. One way is by including representatives of appli- 
cants’ interests in the Board’s membership and governing struc- 
ture. Since there are no explicit organizations for applicants, it 
would be necessary to create appropriate representation in such 
a way that applicants’ interests play a part in Board plans and 
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programs. The second way is by re-creating the College Board 
as a nonmembership organization, responsible to colleges and 
applicants alike— as two sets of clients— but to neither as its gov- 
erning body. In that case, the governance of the Board would be 
independent of both interested parties and would be free to meet 
the needs of both through an assessment of market demands. 

If the first form of reorganization is chosen, the simplest 
method of implementing it is to modify the representation from 
member colleges and secondary schools to include a student 
representation as well as institutional representation. This might 
create the possibility, in the already unwieldy membership meet- 
ings (based on a membership of more than 800 colleges and 300 
secondary schools), of sharp division along interest lines. Such 
direct division need not occur, however, with appropriate struc- 
turing, because there are few if any decisions that must be made 
jointly by representatives of both applicants and institutions. The 
members representing colleges should have power to decide 
what information they require from applicants to aid in their se- 
lection among applicants. The members representing applicants 
should have power to decide what information applicants re- 
quire from colleges to aid in their selection of colleges. Thus 
some activities of the Board should be subject to review by one 
set of members; other activities of the Board should be subject to 
review by the other set of members. 

Restructuring of the Board could be formalized in a some- 
what different way by creating a parent organization, under 
which there would be two subordinate organizations: the pres- 
ent Board with its present membership, and another organiza- 
tion named something like the “College Selection Board” 
whose members represented applicants’ interests. The parent or- 
ganization would not be responsible either to colleges or appli- 
cants, but the two subordinate bodies would be responsible to 
their respective memberships. 

If the second general form of reorganization is chosen, the 
College Board would merely reorganize as a nonmembership 
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organization. It would provide services both to colleges and ap- 
plicants, with the interests of both expressed as interests of 
clients. 

These arc various possible forms that a symmetric governing 
structure could take. Appropriately structured, there is no need 
for confrontation politics or deadlocks between the two major 
interests, because college interests should be the controlling voice 
in certain decisions, while applicant interests should be the con- 
trolling voice in other decisions. 

This structural change is the most essential change of all, to 
insure that the Board develops the appropriate mechanisms for 
acting as a true intermediary between colleges and applicants, 
and to pursue the interests of both in assuring a successful match. 
For while specific recommendations can be made from the van- 
tage point of the present about mechanisms to introduce sym- 
metry into the matching process— such as the first three recom- 
mendations made here— the future will bring new problems that 
can be addressed only if the Board is structurally symmetric in 
the interest of both parties. 

Recommendations 

It is recommended that the College Entrance Examination Board: 

(1) Systematically gather and publish relevant information 
about colleges, the nature of that information to be determined 
by a working group including representatives of colleges, stu- 
dents, and high schools. 

(2) Beyond the standardized testing that allows colleges to 
make direct comparisons of a few characteristics of applicants, 
provide the applicant with the opportunity, through selection 
from a very wide variety of special tests ranging in subject mat- 
ter from numismatics to algebraic topology, to exhibit his special 
capabilities, developed in or out of high school, that may be rele- 
vant to the colleges’ interest in him. 

(3) Provide colleges and applicants with information about 
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the other only if they arc willing to allow information about 
themselves to be made available. 

(4) Modify its governing structure to embody a symmetry 
of college and applicant interests. 

Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
The Principle of Symmetry in College Choice 



It is recommended that the College Entrance Examination Board: 

1. Systematically gather and publish relevant information about 
colleges, the nature of that information to be determined by a 
working group including representatives of colleges, students, 
and high schools. 

Endorsed by 20 members: Abelson, Blocker, Carroll, Coleman, 
Cross, Foster, Friedenberg, Gordon, Hechinger, Herscy, Holtz- 
man, Hoy, Marland, Pace, Pearson, Prentice, Schlossberg, Shir- 
ley, Thresher, Ticdeman. 

Pace: “A lot of the information about colleges will not mean 
anything to high school students or their parents unless it is ex- 
pertly interpreted by the College Board.” 

Prentice: “Who can be ‘representatives of students? I urge 
that college students help describe their colleges and that high 
school students be asked what they wish to know about colleges, 
but I cannot support the idea that these resource people arc rep- 
resentatives in the same (political) sense in which high schools 
and colleges are represented by their officers.” 

2. Beyond the standardized testing that allows colleges to make 
direct comparisons of a few characteristics of applicants, provide 
the applicant with the opportunity, through selection from a 
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very wide variety of special tests ranging in subject matter from 
numismatics to algebraic topology, to exhibit his special capabili- 
ties, developed in or out of high school, that may be relevant to 
the college’s interest in him. 

Endorsed by 18 members: Blocker, Carroll, Coleman, Cross, Fos- 
ter, Fricdcnberg, Gordon, Hersey, Holtzman, Hoy, Marland, 
Pace, Pearson, Prentice, Schlossberg, Shirley, Thresher, Ticde- 
man. 

Foster: “If possible.” 

Pace: “With reservations. It is probably not feasible to de- 
velop a lot of esoteric tests. But one can ask students about such 
special interests and accomplishments. 11 

Prentice: “Numismatics has remained in this brief despite ear- 
lier criticism. I consider the example frivolous and likely to de- 
tract from the value of Coleman’s recommendations. What col- 
lege will care about such matters?” 

Not endorsed by 2 members: Abclson, Hechingcr. 

3. Provide colleges and applicants with information about the 
other only if they arc willing to allow information about them- 
selves to be made available. 

Endorsed by 17 members: Blocker, Carroll, Coleman, Cross, Fos- 
ter, Fricdcnberg, Gordon, Hechingcr, Herscy, Holtzman, Hoy, 
Marland, Pearson, Prentice, Schlossberg, Thresher, Tiedeman. 

Prentice: “Probably there would be no problem about Rec- 
ommendation 3 in practice, but it docs seem that both applicant 
and institution should have a degree of choice about the particu- 
lar information submitted. The recommendation might be in- 
terpreted to mean that a college could be excluded from the 
Board’s services if it declined to provide all information on some 
standard list. It is conceivable that a panel might recommend 
the inclusion of information (salaries, religious composition, po- 
litical affiliations) that a particular college would, on principle, 
decline to provide.” 
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Not endorsed by j members: Abelson, Pace, Shirley. 

Pace: “I doubt if this is necessary. It should be a goal but not 
a prerequisite for action.” 

Shirley: “Blackmail is not needed; colleges will, if reasonable 
requests are made, comply.” 

4. Modify its governing structure to embody a symmetry of 
college and applicant interests. 

Endorsed by 13 members: Blocker, Carroll, Coleman, Cross, Fricd- 
enberg, Gordon, Hcrsey, Hoy, Marland, Pace, Pearson, Pren- 
tice, Schlossberg, Thresher, Tiedeman. 

Pace: “The possibilities for restructuring the College Board 
that Coleman mentions are intriguing, and I would like to sec 
all of them explored. I agree that the structure of the Board 
membership needs to be brought into the twentieth century. 

“The match between applicant and college may or may not 
be successful! The basic gain is that both parties will know that 
they have been involved in the decision.” 

Prentice: “I am much less certain than Coleman is about the 
fundamental importance of structural changes in the Board. 
With rare exceptions we assume that patients’ interests are met 
by physicians and students’ interests by educators. The notion 
that enlightened altruism cannot exist troubles me. I am also 
convinced that very few eleventh- and twelfth-grade students 
know enough about their educational needs to contribute effec- 
tively as voters or policy makers (as distinguished from resource 
people).” 

Thresher: “I think we all agree in principle, but the difficul- 
ties are very great and insufficiently analyzed. The question is 
how.” 

Not endorsed by 3 members: Abelson, Foster, Hcchingcr, Holtz- 
man, Shirley. 

Foster: “Not in any form yet suggested.” 

Holtzman: “Probably impractical to consider, although I 
would endorse it if it can be worked out.” 
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Shirley: “Not practical— either to abandon the Board or to 
have half of the membership transitory.” 

General Comments 

Ahelson: “I agree that a more symmetrical relation between stu- 
dent applicants and institutions is desirable. However, Coleman’s 
Recommendations 3 and 4 go too far. Recommendation 2 is 
desirable but probably impractical.” 

Foster: “I am in favor of better information about themselves 
and colleges going to students, but I have heard no well-in- 
formed student asking for power in the admissions process; this 
being based on insufficient information and experience on the 
part of students, would be chaos. 

“One aspect of the ‘market’ must still be considered for some 
years to come. As long as there is a scarcity of places in the most 
popular colleges and universities we arc in a ‘seller’s market,’ and 
those colleges and universities will inevitably have more ‘power’ 
than their applicants. However, even those institutions should 
cooperate in providing applicants with all possible valid infor- 
mation to facilitate suitable choices by prospective applicants.” 
Marlatid: “There still is an unresolved detail: how actually 
does the student representative to the central governing body 
get identified, selected, elected, and so forth?” 
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CanTesting Contribute to the Quest 
for Community among Students? 

Edgar Z. Friedenberg 



The accepted function of aptitude and achievement testing is to 
identify, select, and appraise the best qualified individual com- 
petitors for membership and preferment within a meritocratic 
hierarchy. So far as I can recall, this has not been seriously ques- 
tioned within the Commission on Tests even by myself, though 
I have fumbled toward expressing my continued dissatisfaction 
with this assumption. 

This assumption, of course, limits and channels consideration 
of the testing function. It permits the Commission to question 
whether the Board is testing the right things— that is, whether 
the aptitudes and achievements measured are socially and indi- 
vidually relevant and appropriate. But it also implies that if the 
present tests arc not testing the right things the most serious and 
unfortunate consequences will be that (i) individual competitors 
will suffer a biased adverse appraisal because their particular 
strengths have not been recognized and validated and (2) candi- 
dates who arc test-wise, or saturated with middle-class patterns 
of response to competitive school situations, will gain an unfair 
advantage in scoring higher than their actual aptitudes warrant; 
they will become “ovcrachievers.” 

The efforts of testing agencies, consequently, become directed 
toward increasing the heterogeneity of rcwardablc response, to 
avoid forming a test-wise “elite” from which students with 
other equally or more admirable qualities are excluded. This is a 
commendable concern, when one considers what such an elite is 
like. The universal use of short-answer, standardized tests as ar- 
biters of advancement has undoubtedly contributed to the domi- 
nance of the empirical, fact-oriented cognitive style that leaves 
our society unable to respond to necessarily ambiguous aesthetic 
and moral judgments. Conversely, a people dedicated to com- 
petition and manipulation surely find their social preferences 
sustained by the kind of testing that has evolved. Granted the 
College Board, Robert McNamara follows. 

This docs not, of course, mean that testing, or the College 
Board, cannot serve quite different functions. We do not, how- 
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ever, as a Commission, seriously consider really different possi- 
bilities. We worry about whether the Board’s tests are keeping 
the underclass out of the American poker game, and search for 
procedures that will deal them in— there must be some kind of 
test they would do better at than middle-class kids do. But the 
game is not so much a poker game as a crap game, and some 
young people have decided not to take any more crap. Most of 
our effort seems to me directed at keeping the underclass from 
contributing to their number; we want to make sure the middle 
class continues to run the only game in town, even if this means 
finding ways of involving in it people who really don’t want or 
know how to play. If they will only consent to train for com- 
petition, we will find some way for them to experience success 
—at least in school. 

I am far more interested, however, in seeing the College 
Board respond to the thrust of student discontent that is directed 
against the crap game itself, and against the assumption that the 
most serious function of a college is to maintain an arena in 
which individuals can compete against one another under handi- 
caps designed to compensate for the unfair advantages of class. 
Education is shared experience, and only that; and the less con- 
tinuity there is among those who share an educational institu- 
tion, cither between themselves or between their present and 
their past, the less profound and intense the educational experi- 
ence is likely to be. There is, in short, something to be said for 
the educational use of tests that group together people who have 
enough in common to allow them to establish a community. 
The difficulty lies in what they have in common; tests that select 
out for a common reward people whose special excellence is in 
competition and the fragmentation of experience arc not going 
to bring together a group who arc likely to enrich one another, 
or to care for those dependent on their leadership. 

I would therefore like to sec the Board consider offering not 
merely achievement and aptitude tests but diagnostic tests that 
would permit colleges and students to select each other with 
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due regard for prospective compatibility. There are many de- 
vices that might be used for this, but the most direct, involving 
least invasion of privacy, would be a paper and pencil group in- 
strument designed to furnish a profile in terms of the categories 
developed by Henry Murray for the TAT (Thematic Appercep- 
tion Test), along with an institutional typology of the sort de- 
vised by George Stem at Syracuse University for assessing 
institutional climates. Together, these would supply enough in- 
formation to allow both the student and the college to anticipate 
the degree of fit between them. Of course the most congruent 
combination should not be chosen — the desired amount of dis- 
cord could be intentionally sought in the most expressive possi- 
ble combinations. In making this proposal I do not mean to 
suggest that congruence between persons and institutions is de- 
termined only by psychological considerations but that relevant 
social-class variables would express themselves as psychological 
differences— otherwise they would hardly be relevant. A liberal 
and permissive school ought, I believe, to be able to defend itself 
against an influx of punitive, authoritarian, or extremely passive- 
dependent people, but it ought not to be permitted to exclude 
any social class or ethnic group as such. The psychological in- 
strument, so long as it is permitted to retain its natural cultural 
contamination, should be quite sufficient. 

As a measure totally unrelated to the above proccdurally, but 
still directed toward the end of contributing to the quest for 
community, I urge that the Board offer high school juniors and 
seniors, or college students intent on transfer or graduate study, 
the opportunity to ta/.e tests for the purpose of obtaining scores 
intcrprctablc by reference to national norms, in lieu of grades 
and, if legally possible, in lieu of school attendance. In this way 
certain kinds of people who possess and can display quite con- 
ventional forms of achievement, but who arc likely to prove un- 
acceptable to school personnel and to be forced to accept either 
poor credentials or slow stultification, could hope to preserve 
themselves intact. This procedure is, in effect, the converse of 
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the usual proposal for seeking new kind of tests for identifying 
talent among the “culturally deprived,” but still subjecting them 
to institutional discipline and endorsement before recognizing 
their gifts. The consequence of my proposal would be to limit 
society’s suzerainty to those skills and traits it could openly and 
officially demand as requirements, while preventing the imple- 
mentation of informal demands that, if expressly made, could 
not readily be defended. My conviction that this would contri- 
bute to community among students is based on my belief that in- 
formal institutional demands of the schools arc usually explicitly 
dehumanizing and that the best protection for the civil liberties 
of students is to be found in distinguishing sharply between 
society’s right to demand certain skills of every citizen, and its 
practice of requiring that these be exhibited as aspects of a pe- 
dantic and constricted pattern of tastes as well. 

Recommendations 

It is recommended that the Colbg^ Entrance Examination Board: 

1. Offer not merely achievement and aptitude tests but also 
diagnostic tests that would permit colleges and students to se- 
lect each other with due regard for prospective compatibility. 

2. Offer high school juniors and seniors, or college students 
intent on transfer or graduate study, the opportunity to vkc tests 
for the purpose of obtaining scores, intcrpretable by reference to 
national norms, in lieu of grades and, if legally possible, in lieu of 
school attendance. 






Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
Can Testing Contribute to the Quest 
for Community among Students? 

\ 

It is recommended that the College Entrance Examination Board: 

i. Offer not merely achievement and aptitude tests but also diag- 
nostic tests that would permit colleges and students to select each 
other with due regard for prospective compatibility. 

Endorsed by 17 members : Blocker, Carroll, Coleman, Foster, 
Fricdcnberg, Gordon, Hechinger, Herscy, Holtzman, Hoy, 
Marland, Pace, Pearson, Schlossberg, Shirley, Thresher, Tiedc- 
man. 

Foster: I endorse the development of such tests for use only 
by experienced and well-trained counselors with students. I do 
not endorse their use by colleges to select students, a practice that 
would be difficult if not impossible; the use, for admissions pur- 
poses, by colleges of “personality tests” would be unsuitable for 
private colleges and inapplicable for public. 

Hechinger: “My endorsement is with some reservations— 
largely based on my concern that colleges might become too 
homogeneous in composition and purpose. Thus, I support the 
principle of Recommendation v only if assurance of breadth of 
selection and safeguards against ideological domination (political 
as well as educational) can be provided.” 

Shirley: “Detached from the inflammatory, truculent justifi- 
cation given by Friedenbcrg, this recommendation is reasonable. 
There is some indication that he may feel the tests would have 
use for determination of nonacademic (hippie) goals, but even 
that might not be a bad thing for both students and colleges.” 

Not endorsed by 2 members: Cross, Prentice. 
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i member abstained: Abelson. 

2 . Offer high school juniors and seniors, or college students in- 
tent on transfer or graduate study, the opportunity to take tests 
for the purpose of obtaining scores, interpretable by reference to 
national norms, in lieu of grades and, if legally possible, in lieu of 
school attendance. 

Endorsed by 17 members: Blocker, Coleman, Foster, Friedenberg, 
Gordon, Hechinger, Hersey, Holtzman, Hoy, Marland, Pace, 
Pearson, Prentice, Schlossberg, Shirley, Thresher, Ticdeman. 

Foster: “Endorse, except for the phrase ‘and, if legally possi- 
ble, in lieu of school attendance/ This ignores all educational 
gains except those testable, as in a formal course. Although I, too, 
hope for some ‘credit’ allowed, as Friedenberg suggests, a col- 
lege education will lose much of its value if it comes to provide 
only testable subject matter and academic skills.” 

Hersey : “There seems to be some danger that in the end it 
would be the achievers, those who have the habit of quickness 
in mastering and besting the system, who would benefit most 
from the provision of such tests and scores, rather than the mav- 
ericks and independents whom the recommendation is intended 
to protect.” 

Schlossberg: “This brief was fun to read: lively, pointed, stim- 
ulating. And because Friedenberg has such a fresh approach I 
hesitate to criticize. However, much as I agree with Frieden- 
berg’s criticisms of the schools, I do not feel that a brilliant 14- 
year-old who can pass all the examinations would psychologi- 
cally be ready for college. Despite the ‘dehumanizing’ effects of 
school, this is better than placing a brilliant youngster in a social 
situation with which he is not ready to cope.” 

Not endorsed by 2 members: Carroll, Cross. 

Carroll: “I really do not understand this recommendation. 
Perhaps I could if it were better spelled out. In any case, one im- 
plication of the recommendation to which I would probably ob- 
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ject is that the College Board would in effect offer tests that 
would substitute for examinations given by schools. It seems to 
me that such a system would go beyond the Board's scope, and I 
don't see how it would achieve the objectives Friedenberg men- 
tions.” 

i member abstained: Abelson. 

General Comments 

Marland : “I concur with this set of recommendations, but for 
reasons other than those exhibited in the text. The notion of 
compatibility between individuals and institutions leaves me 
with doubts, both as to the wisdom of a 17-year-old in assessing 
the reported vagaries of institutional character and the assumed 
competence and integrity of institutions of higher learning to 
orchestrate ‘the desired amount of discord.' One possible out- 
come of the proposed arrangement would be that institutions of 
higher learning would become all blue or all green or all mauve 
(that would be like beige institutions, calling for a common color 
made up of other, original colors, chemically or physically trans- 
formed). 

“The brief seems to say that once a student reaches college, 
having suffered the damaging am! dehumanizing influences of 
the schools long enough, all will be well. My own observations 
suggest that schools, within the unhappy protocols of present ad- 
missions testing systems, extend themselves to the point almost 
of perjury to advance young people to higher education, both in 
spirit (aspiration) and in fact (entry). However, I think the direct 
route to college by way of national-norm instruments is valid for 
some. Most 16- and 17-year-olds need more growing up psy- 
chologically at home, however, before being cast upon the 
doubtful mercies of the college system. 

“This paper implies, on page 2, that all the elementary-sec- 
ondary world orbits around tests (empirical, fact-oriented, and so 
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forth) leaving no space for aesthetic and moral growth. This 
overstates the influence of tests and understates the meaning of 
public schools. 

“Nevertheless, for good reasons, I endorse these recommen- 
dations.” 

Pace: “I endorse the principle of this proposal, but I reject the 
methods proposed to carry it out and also reject the ideology or 
assumptions that seem to lie behind these methods.” 

Prentice: “The statements that ‘education is shared experi- 
ence, and only that’ and that there is something to be said for 
grouping together ‘people who have enough in common to al- 
low them to establish a community’ arc expressions of a highly 
personal educational philosophy that deserves elaboration and 
justification. Needless to say, it could be interpreted as a justifi- 
cation for complete educational separation among social classes, 
and I doubt that the Commission would support such a radical 
view without strong empirical support for it.” 

Thresher: “I endorse these recommendations because they cut 
through much of the artificiality of current practice, but I recog- 
nize that they will bring other problems. This, like other briefs, 
points out a direction for experiment, not a proposal for abrupt 
action. 

“I tli ink Fricdenbcrg overstresses the possibilities of artificially 
arranged compatibility through the medium of a ‘composed' 
group. This practice in itself is highly manipulative and anti- 
Fried enbergian. It implies the legitimacy of classifying people 
and determining their destinations accordingly. It would seem, 
rather, that compatibility is best brought about by allowing and 
helping kindred spirits to find each other in the midst of a heter- 
ogeneous multitude. There will be more such, as the average 
size of institutions increases, even though there will still be ‘spe- 
cialty shops' (Riesman’s phrase) such as St.John’s, Bennington, 
and Caltech. 

“Maximum diversity is, in itself, immensely educational. The 
idea of a small, congenial minority happily projecting its own 
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special perspectives on the world, unchecked by any real diver- 
sity of skeptics, does not strike me as good education. If we are 
to protect the kind of people Friedenberg likes from those Mc- 
Namara likes, we are in duty bound also to do the converse. 
Thus both groups in a very real sense suffer a curtailment of edu- 
cational breadth. 

“Friedenberg almost but not quite questions the divine right 
of every college to select its own students— a right taken for 
granted in all these briefs. I could hope for some stress on the 
obligation of colleges to accept a reasonable cross section of the 
public, and their willingness to be judged by what they can do 
with such a group, without the adventitious aid of prior super- 
selection. ” 
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Toward a Qualitative Approach 
to Assessment 

Edmund W \ Gordon 



Much of the impetus for the development of a technology of as- 
sessment related to intellective function and achievement re- 
sulted from and has been maintained by a supply-and-demand 
approach to access to education and distribution of educational 
opportunities. Access to a limited supply of educational oppor- 
tunities has been guarded by selection procedures that prior to 
the twentieth century were based on the prospective student’s 
social status. In the pre-Rcformation period access to education 
was limited to the political and religious nobility and later to 
other privileged classes, while the twentieth-century selection 
procedures have come to be dominated by the student’s demon- 
strated or predicted intellectual status. Where the supply of op- 
portunities has been limited great emphasis has been placed on 
the selecrion of students and the prediction of their performance 
when exposed to those opportunities. Binet’s work in intelli- 
gence-test development was directed toward the creation of an 
instrument that could be used to identify those pupils who were 
likely to benefit from schooling. His admonitions that education 
also turn to treatment of those exposed as not likely to succeed 
were generally ignored. In a period of scarce educational oppor- 
tunities, Binct’s concern for the educability of ; *'Hligcnce did 
not gain favor. Society found greater utility in promise of 
the predictive and selective validity of his new test. 

This emphasis on selection and prediction has continued even 
though the social conditions that gave rise to it have changed. 
In recent years, we have seen in America a growing concern 
with universal access to secondary and higher education. The 
educational requirements of the nation are increasingly defined 
as post-high school educational opportunities for almost all 
youth and continued learning for most people. If this trend con- 
tinues, selection and prediction can no longer be allowed to 
dominate in the tcchnoh gy of psychoeducational appraisal. 
Rather, the stage must be shared with an emphasis on description 
and prescription— that is, the qualitative description of intellective 
function leading not to the selection of those most likely to suc- 
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ceed but to the prescription of the learning experiences required 
to more adequately insure that academic success is possible. 

Psychological testing obviously can be used to measure 
achieved development. From those achievement patterns, subse- 
quent achievement in the same dimensions of behavior under 
similar learning-experience conditions can be predicted with 
reasonable validity. Thus people who have learned an average 
amount during one learning period (high school) may be ex- 
pected to learn an average amount in the next learning period 
(college). However, adequate attention has not been given to the 
facts that psychological testing can be used to describe and quali- 
tatively analyze behavioral function to better understand the 
processes by which achievement is developed, to describe non- 
standard achievements that may be equally functional in subse- 
quent situations requiring adaptation, or to specify those condi- 
tions in the interaction between learner and learning experience 
that may be necessary to change the quality of future achieve- 
ments. 

In the present situation confronting those concerned with 
access to higher education for larger numbers of young people 
and for youth from more diverse backgrounds than those from 
which college students were previously chosen, it is not enough 
to simply identify the high-risk students. The tasks of assessment 
and appraisal in this situation are to identify atypical patterns of 
talent and to describe patterns of function in terms that lead to 
the planning of appropriate learning experiences. It is therefore 
recommended that the College Entrance Examination Board 
immediately: 

l. Explore possibilities for adding to its quantitative reports 
on the performance of students, reports descriptive of the pat- 
terns of achievement and function derived from the qualitative 
analysis of existing tests. The Board’s existing instruments 
should be examined with a view to categorization, factorial an- 
alysis, and interpretation to determine whether or not the data 
of these instruments can be reported in descriptive and qualita- 
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tive ways, in addition to the traditional quantitative report. 

For example, response patterns might be reported differen- 
tially for 

A. Information recall 

(1) Rote recall 

(2) Associative recall 

(3) Derivative recall 
or 

B. Vocabulary 

(1) Absolute 

(2) Contextual 

2 . Explore the development of test items and procedures that 
lend themselves to descriptive and qualitative analyses of cogni- 
tive and affective, adaptive functions, in addition to wider spe- 
cific achievements. 

A. In the development of new tests, attention should be giv- 
en to the appraisal of 

(1) Adaptation in new learning situations 

(2) Problem solving in situations that require varied cog- 
nitive skills and styles 

(3) Analysis, search, and synthesis behaviors 

(4) Information management, processing, and utilization 
skills 

(5) Nonstandard information pools 

B. In the development of new procedures, attention should 
be given to the appraisal of 

(1) Comprehension through experiencing, listening, and 
looking, as well as reading 

(2) Expression through artistic, oral, nonverbal, and 
graphic, as well as written symbolization 

(3) Characteristics of temperament 

(4) Sources and status of motivation 

(5) Habits of work and task involvement under varying 
conditions of demand 

C. In the development of tests and procedures designed to 
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get at specific achievements, attention should be given to 

(1) Broadening the varieties of subject matter, compe- 
tencies, and skills assessed 

(2) Examining these achievements in a variety of con- 
texts 

( 3 ) Open-ended and unstructured probes of achieve- 
ment to allow for atypical patterns and varieties of 
achievement 

(4) Assessing nonacademic achievements such as social 
competence, coping skills, avocational skills, and ar- 
tistic, athletic, political, or mechanical skills 

3. Explore the development of report procedures that con- 
vey the qualitative richness of these new tests and procedures to 
students and institutions in ways that encourage individualized 
prescriptive educational planning. What is called for is a state- 
ment abou ac nature of adaptive function in each individual 
that lends its If to planning a way of intervening in and facilitat- 
ing his development. Patterns of strength and weakness, condi- 
tions conducive to successful coping, conditions resulting in 
congruence and engagement or incongruence and alienation are 
examples of the kind of information required. 

4. Explore the development of research that will add to 
understanding of the ways in which more traditional patterns of 
instruction will need to be modified to make appropriate use of 
wider ranges and varieties of human talent and adaptation in 
continuing education. It would be relatively useless to identify 
broader ranges of behavior if these did not hr»ve their represen- 
tation in programs of instruction, and if opportunities for the use 
of these adaptive patterns in learning were not available to 
young people. Alongside modification of instruments of assess- 
ment and of procedures for appraisal there needs to be a consid- 
erable amount of attention given to modifying the curriculum 
and conditions under which teaching and learning occur. 

It must be recognized that the proposals advanced in this 
brief arc, at the present stage of development in human ap- 
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praisal, very much in the conceptual stage. There is some re- 
search but little completed work that can be used to implement 
such a program. Yet a serious commitment to the achievement 
of symmetry in the continuing-education entry process, to the 
broadening of opportunities for access to continuing education, 
as well as to the greater adaptation of continuing-education 
programs to the requirements of the extremely varied popula- 
tions to be served— this commitment demands that answers be 
provided to the problems implicit in these proposals. 

Recommendations 

It is recommended that the College Entrance Examination Board: 

1. Explore possibilities for adding to its quantitative reports 
on the performance of students, reports descriptive of die pat- 
terns of achievement and function derived from the qualitative 
analysis of existing tests. 

2. Explore the development of test items and procedures that 
lend themselves to descriptive and qualitative analyses of cogni- 
tive and affective adaptive functions, in addition to wider spe- 
cific achievements. 

3. Explore the development of report procedures that con- 
vey the qualitative richness of these new tests and procedures to 
students and institutions in ways that encourage individualized 
prescriptive educational planning. 

4. Explore the development of research that will add to 
understanding of the ways in which more traditional patterns of 
instruction will need to be modified to make appropriate use of 
wider ranges and varieties of human talent and adaptation in 
continuing education. 
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Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
Toward a Qualitative Approach to Assessment 



It is recommended that the College Entrance Examination Board: 

1. Explore possibilities for adding to its quantitative reports on 
the performance of students, reports descriptive of the patterns 
of achievement and function derived from the qualitative analy- 
sis of existing tests. 

Endorsed by 18 metttbers: Abelson, Blocker, Carroll, Coleman, 
Foster, Fricdcnbcrg, Gordon, Hersey, Holtzman, Hoy, Marland, 
Pace, Pearson, Prentice, Schlossberg, Shirley, Thresher, Ticde- 
man. 

2. Explore the development of test items and procedures that 
lend themselves to descriptive and qualitative analysis of cogni- 
tive and affective adaptive functions, in addition to wider spe- 
cific achievements. 

.Endorsed by t8 members : Abelson, Blocker, Carroll, Coleman, 
Foster, Friedcnberg, Gordon, Hersey, Holtzman, Hoy, Marland, 
Pace, Pearson, Prentice, Schlossberg, Shirley, Thresher, Ticde- 
man. 

3. Explore the development of report procedures that convey 
the qualitative richness of these new tests and procedures to stu- 
dents and institutions in ways that encourage individualized 
prescriptive educational planning. 

Endorsed by 18 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Fricdcnbcrg, Gordon, Hersey, Holtzman, Hoy, Marland, 
Pace, Pearson, Prentice, Schlossberg, Shirley, Thresher, Ticdc- 
man. 
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4. Explore the development of research that will add to under- 
standing of the ways in which more traditional patterns of in- 
struction will need to be modified to make appropriate use of 
wider ranges and varieties of human talent and adaptation in 
continuing education. 

Endorsed by 18 members : Abelson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Hcrsey, Holtzman, Hoy, Marland, 
Pace, Pearson, Prentice, Schlossberg, Shirley, Thresher, Tiede- 
man. 

Foster: '‘This needs to be done, but the Board is not the 
proper agent to do it; it would be presumptuous for the Board to 
do much research in this area.” 

General Comments 

Carroll: “This is an excellent statement, and I find that it accords 
quite well with my own ideas as to the ways in which research 
might be directed to yield more meaningful procedures whereby 
student characteristics are reported. 

“There may be some semantic difficulty with the word 
qualitative. Any quantitatively expressed score can be said to 
have a qualitative aspect as soon as it is accompanied by a state- 
ment about the kind of function being measured. Already the 
sat score reports imply qualitative information in the sense that 
they refer to two kinds of cognitive performance— verbal and 
mathematical. This concept could be extended to the various 
cognitive and affective functions alluded to by Gordon— that is, 
the scores would be quantitatively expressed but would be iden- 
tified in qualitative terms. I would want to know whether Gor- 
don accepts this interpretation. 

“Anything more qualitative than what I have suggested 
would take one into the realm of purely verbal description. Sev- 
eral years ago Carl Helm at ets experimented with computer- 
written sat reports that interpreted the quantitative scores in 
purely verbal, qualitative terms. The machine would write a 
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prose paragraph based on the student’s scores and other informa- 
tion. Perhaps Helm’s technique could be extended to a much 
wider variety of test performances, but one would have to avoid 
the danger of making the resulting prose appear too mechanical 
and stereotyped. I wonder whether this kind of computerized 
score report would be satisfactory to Gordon.” 

Hersey: “If there were to be a rank order of undertakings 
proposed in various briefs of the Commission, I would place 
those proposed here very high on the list. 

“Under (2), B, 3 and 4, it seems to me that a most vital ap- 
praisal would be of something within the categories of ‘charac- 
teristics of temperament’ and ‘sources and status of motivation,’ 
and combining them— namely, the factor of persistence, called 
‘drive,’ the combination of interest and mobilized energy, which 
will not die and refuses to be denied. It is the underlying force of 
most achievement, and it is something that includes but is differ- 
ent from motivation.” 

Hoy: “A significant paper that supports die notion that the 
present tests are becoming not only irrelevant to the educational 
needs of American students but alsc obsolete. 

“The brief significantly addresses itself to what must be done 
to change testing so that it can regain a contributory role in edu- 
cation. The problem emphasized here is the need for a new ap- 
proach to testing that is neither destructive of the testing estab- 
lishment nor utopian in vision, but rather a realistic way to 
move the College Board into more significant areas of endeavor. 

“In several ways this brief sums up the central theme of the 
Commission’s deliberations, especially Recommendation 4.” 
Marlartd: “The first three recommendations, in addition to 
calling for major reform in the College Board, call for a corre- 
sponding reform on the part of admissions officers in colleges. 
Should some attention be given to this in the brief?” 

Prentice: “I think we are years away from useful instruments 
of this type. ‘Exploring’ such developments is fine, but some 
limits should be set to the open-ended search for useful measures. 
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Dollars? Time? I am not sure, but there should be some kind of 
cost-benefit analysis.” 

Shirley: “All these recommendations call for research, and 
this is certainly badly needed. Even with the answers in hand, 
however (and those may be years off), the implementation of 
these new educational ambitions would call for rather drastic 
changes in the educational (and social) patterns of our nation. 
This may be desirable, but the task is difficult.” 

Thresher: “These are all valid long-range goals, and they re- 
flect Gordon’s admirable insistence on educational goals that can 
be clearly defined and methodically arrived at and measured, in 
contrast to sporadic experiments too soon abandoned. But in the 
long, intervening period there will have to be a great deal more 
resort co loosening and broadening entrance requirements. Some 
rough-and-ready randomization of the admissions process will 
be needed, as a practical expedient. The Commission’s report 
should reflect the concern to mitigate excessive and ill-directed 
selectivity.” 



A College Entrance Service 

John Hersey 



This brief proposes a service, presumably to be comprised of 
numerous machines and human beings, that would provide an 
approach to true symmetry in the process of entrance to higher 
education. 

The ideal of such a symmetry depends upon the adequacy, 
relevance, and symbiotic usefulness of information available to 
the two parties to the transaction. I do not believe that symmetry 
can be achieved to the satisfaction of either party by providing 
to the college, on the one hand, personal and test data on the in- 
dividual applicant, and to the student, on the other hand, cither 
merely a detailed description of colleges to which he has already 
decided to apply, or, wanting such a decision, an overwhelm- 
ingly formidable description of the more than 2,000 institutions 
in the United States. 

It should be possible to devise an exchange in which both the 
inputs and the outputs would be more nearly balanced and more 
explicitly useful in the process of choice than that. The recipro- 
cal goals of the partners in this exchange would be for the stu- 
dent to find the handful of colleges, and eventually the one 
college, that would most nearly fit his needs and desires and tal- 
ents, and for the college to find the students who, along with its 
faculty, would enable it to become a community manifesting its 
own best possibilities. 

Inputs 

Student input to the service proposed here would consist of a 
complex of results of tests of the sorts the Commission on Tests 
has discussed— aptitude, achievement, and diagnostic tests “that 
would permit colleges and students to select each other with due 
regard for prospective compatibility” (Friedcnberg brief, page 
33); guidance and reference comments on the student; high 
school records; descriptive and evaluative material on the high 
school and community from which the student comes; and 
something from the student about his own view of himself. An 
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attempt should be made to elicit from the student not only his 
special interests and bents, insofar as they are developed, but also 
some sense of his goals, his style of life, his sources of energy, and 
even his hangups, insofar as he is willing to share them. 

College input would consist of information supplied by the 
institution; assessments of the institution by students, faculty, 
administrators, and students’ parents, including results of mea- 
sures of sorts described in Pace’s brief (pages 87-99); and assess- 
ments of the institution by high schools that feed it students. 
This input would include “statistics on test- score distributions of 
entering freshmen; the dropouts and transfers, by cause, in each 
year, including the proportion of entering freshmen who com- 
plete college in the normal period; the distribution of types of 
degrees awarded; the proportion of graduates going on to pro- 
fessional schools, graduate schools, and various types of occupa- 
tion; standardized statistics on distribution of class size, on num- 
ber of class hours of teaching per year per faculty member, and 
faculty-student ratio; a measure obtained from student ques- 
tionnaires of the social and intellectual climate and focus of ac- 
tivities of the student body” (Coleman brief, pages 19-32). It 
should also include, among numerical data, the size of the col- 
lege, fees charged, financial aid practices, and geographic and 
sex distributions of students. Besides, more qualitative informa- 
tion of interest to potential students should be included (here I 
amend the Coleman formulation, going into far more detail in 
this section than in the one on student input because there seems 
to be far more extensive experience in testing and describing 
students than in describing and evaluating colleges): information 
about the principal emphasis of the institution (liberal arts, en- 
gineering, and so forth); academic prerequisites; the nature of 
special programs and activities offered by the institution; com- 
pensatory programs, if any, for culturally deprived students; de- 
scription of housing; opportunities for seminars and tutorial 
work, and for work-study experiences; the degree of flexibility 
of the course system and the grading system; major fields of 
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study offered, and requirements for major and minor course 
elections; religious affiliation, if auy; the degree of flexibility as to 
the time required for the baccalaureate degree; a description of 
the setting of the college, whether urban or rural, and an ac- 
count of its relationship with the community in which it exists; 
description of institutional governance, and machinery and prac- 
tice of student participation, if any, in the decision-making proc- 
ess; evaluative material on academic departments; citations, by 
field of specialty, of particularly distinguished scholars and teach- 
ers on the faculty. 

Outputs 

The outputs to both parties would be of two sorts. 

To the college, the output would be, first, an evaluation of 
the institution as to the statistical and descriptive items just 
listed, placed in perspective with comparable data about colleges 
with which it is competitive; second, a list, culled from the pool 
of candidates who have taken the College Entrance Examination 
Board tests, of applicants whose qualities and aptitudes might be 
considered particularly apt for the institution, and, with respect 
to the individual candidate, a legible and complex nonnumcrical 
description of a human being, which would give a sense of abil- 
ities, of preparation, of interests, of motivation, of character, and 
of style— in short, of a whole person. 

To the student, the output would be, first, a full description 
of him as a human being in relation not only to choice of col- 
leges but also to a range of career and course and activity choices 
that might be most appropriate for him in the future— in other 
words, informed diagnostic material; second, full descriptions of, 
let us say, 6 to io institutions (whether or not members of the 
College Board) that could best serve his needs and abilities. 
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Can It Be Done? 

At present there is to my knowledge no service that supplies the 
ambitious two-way functions proposed in this brief. The Com- 
mittee on Guidance of the College Board has already been ex- 
ploring existing one-way services , 1 of which there are at least 
seven that are computerized, that have been called, as a group, 
collegolocater services, and that attempt to provide assistance to 
students in narrowing the universe of higher educational institu- 
tions to a reasonable number of choices for the student to ex- 
plore further; as well as a few that arc not computerized. Evi- 
dence has been assembled about seven computerized services: 
Interactive College Suggesting System; the service operated by 
ets for the National Scholarship Service and Fund for Negro 
Students (nssfns), and five of the leading mail order services 
that provide lists of colleges on the basis of questionnaires com- 
pleted by students. 

“The Interactive College Suggesting System (less) operated 
by Interactive Learning Systems, Inc. (ns), is the only opera- 
tional service in which the student has direct access to a data 
bank. In general it does not suffer from the flaws evident in other 
computerized services. It places its services in a larger guidance 
context, one that is judged to be reasonably consistent with cur- 
rent theories of vocational development. Although it docs not 
provide a rationale for its list of . . . characteristics recorded in 
the data bank, it may be that school personnel explain this to stu- 
dents using the service. Its data bank is generally limited to those 
characteristics on which factual information is readily available. 
There is no need to inform the user of how his specifications arc 



i. “College Locatcr Services.” Unpublished report prepared for the Commit- 
tee on Guidance of the College Entrance Examination Board. In this version 
of my brief I have quoted from the report as indicated and have drawn on the 
report for much information about the flaws found in most of the services and 
the advantages and disadvantages of the two kinds of services that might be 
developed.— J.H. 
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manipulated; the student provides the specifications and any 
changes in them. The chief drawback to the service is that it re- 
quires the availability of supporting hardware for the service. . . . 

“The nssfns program is designed to serve a particular pur- 
pose and a particular clientele. It has an operational structure in- 
volving schools and nssfns, unlike any other computerized 
service utilizing a queit.ionnaire-batch-processing system. The 
specifications come largely from the sponsoring organization, 
and feedback to the system is routinely available from counselors 
and institutions.” 

The five remaining services tend to suffer from three or more 
of the following flaws, according to the report: the service tends 
to indicate that the list of institutions provided is the answer to 
the student’s quest rather than merely a point of departure for 
further investigation; docs not provide a rationale for the input 
criteria offered to the user; does not provide an opportunity for 
the user to rank criteria in order of their importance to him; does 
not inform the user of the resources for its own data bank; does 
not provide for frequent systematic updating of its data bank or 
fails to inform the user of such procedures; gives the student an 
opportunity to choose among the characteristics of colleges 
which, if taken, depends upon qualitative judgments about col- 
leges; docs not tell the user how it manipulates the variables in 
arriving at a list of institutions for consideration; arbitrarily re- 
jects criteria the user has specified as important to him but fails 
to provide any explanation of this fact; docs not distinguish in its 
output between the institutions that meet all the criteria the user 
has designated and those that possess only some of cnese charac- 
teristics; has no recognized, representative professional group (or 
no group at all) as an advisory or managing body and has no 
cited procedure for evaluation and research. 

A computerized service could apparently be developed that 
would not possess these characteristic shortcomings. There 
would be two basic alternatives in developing such a service: so- 
called batch processing of standard multiple-choice question- 
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naircs to be filled out by the applicant; or an interactive service, 
in which the student, in effect, converses with the computer. I 
will not go far in discussion of these technical matters, except to 
say that the former would be cheaper, would be similar to other 
services ets already provides, would be highly standardized and 
therefore probably safer; but by the same token it would be 
relatively rigid, with less opportunity for the student to take ini- 
tiative, to probe, and to consider alternatives. The standardized 
questionnaire is liable to be perceived by the student as being 
something like an adversary test; he is the passive party in the 
process. The full symmetry that I am proposing in this brief 
would not be possible with the questionnaire system. The inter- 
active system lias the advantages of letting the candidate explore 
options and consider alternatives; it lends itself easily to produc- 
tive repeated interviews, for responses arc stored and remem- 
bered; the student is the active agent. But the system would be 
more expensive, both to develop and to operate, and it would 
require hardware, cither mobile or permanent. 

I would urge the College Board to reject the cheaper, easier, 
and safer course of entering the collcge-locatcr-servicc field ex- 
clusively through a questionnaire-based system. This would 
leave the more flexible and open interactive-system possibilities 
to outfits far less able to build upon experience, responsibility, 
and reputation than the Board and ets, which have already ac- 
cumulated those assets; indeed, there is already evidence of the 
scandalous danger of leaving the field to money-makers, quacks, 
and short-cutters. 

But even more serious from my point of view would be the 
failure to grasp and develop the full implications of the idea of 
symmetry that has been brought forward by the Commission on 
Tests. While the two-way College Entrance Service that I have 
attempted to describe would be enormously difficult and perhaps 
cnonnously expensive to develop, it seems to me not beyond the 
limits either of the imagination of the technicians now available 
to ets and the Board, or of die technology as it already stands. 
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It could provide a tremendous step forward from relatively hap- 
hazard to relatively informed and mutually beneficial choices of 
students by colleges and colleges by students. I need not stress 
the urgency of this decision, for the opportunists arc already at 
work devising inadequate solutions, and students arc wild and 
colleges arc desperate. 

Simultaneous Operations and Research 

This urgency speaks to the need for simultaneous development 
of both operations and research, and for the use of both printed 
and computerized techniques. To wit: 

First, a limited two-wav service based, so far as college input 
is concerned, upon printed materials to be assembled from the 
large mass of “hard” information about colleges that is already 
available, with step-by-step introduction of evaluative and quali- 
tative measures as they arc refined through research. What is 
desperately needed here is something more useful to the student 
than a massive compendium— concretely, a categorizing and in- 
dexing device, subtle enough to be tolerably reliable yet simple 
enough to be manipulated by an unsophisticated applicant, a de- 
vice with which he can narrow the universe of colleges on his 
own initiative, digging out for himself from the bewildering 
mass at least preliminary information about the relatively small 
number of colleges that fit his needs. 

Second, and simultaneously, research of the sort urged in the 
Pace brief (pages 87-99). 

Third, and simultaneously with the others, operational work 
with computerized services. Radicr than developing a service 
from scratch, the Board might purchase one of the existing col- 
lcgc-locatcr services, which might then be carried on and refined 
by stages as research would dictate. 

These more or less headlong procedures would be justified 
on the ground that an imperfect symmetry would be better than 
the present gross asymmetry; and the expense and difficulties in- 
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volved in moving on both operations and research at once would 
be justified by the relatively rapid realization, by these means, of 
a truer symmetry, and by the probability that while rigorous re- 
search would keep operations “honest,” the very fact of on-go- 
ing operations would force the Board to focus on substance and 
content while, and not after, wrestling with the immense tech- 
nical problems inherent in the challenge of total symmetry. 

Recommendations 

It is recommended that the College Entrance Examination Board: 

1. Carry the principle of symmetry to its full realization by 
establishing as soon as possible a College Entrance Service, the 
purpose of which would be: 

a. to guide applying students to colleges that would fit their 
needs and abilities; 

b. to assist institutions in building student populations that, 
along with their faculties, would enable them to become com- 
munities manifesting their own best possibilities; 

c. to supply diagnostic and guidance materials on themselves 
to individual students; 

d. to supply evaluative materials on themselves to colleges. 

2. To this end to undertake simultaneously both operations 
and research, using both printed and computerized techniques. 



Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
A College Entrance Service 



It is recommended that the College Entrance Examination Board: 

i. Carry the principle of symmetry to its full realization by estab- 
lishing as soon as possible a College Entrance Service, the pur- 
poses of which would be: 
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a. to guide applying students to colleges that would fit their 
needs and abilities; 

b. to assist institutions in building student populations that, 
along with their faculties, would enable them to become com- 
munities manifesting their own best possibilities; 

c. to supply diagnostic and guidance materials on themselves 
to individual students; 

d. to supply evaluative materials on themselves to colleges. 

Endorsed by 18 metnbers: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Hechinger, Hersey, Hoitzman, 
Hoy, Marland, Pace, Pearson, Prentice, Schlossberg, Thresher, 
Tiedemau. 

Not endorsed by i member: Shirley. 

2. To this end undertake simultaneously both operations and re- 
search, using both printed and computerized techniques. 

Endorsed by 18 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Hechinger, Hersey, Hoitzman, 
Hoy, Marland, Pace, Pearson, Prentice, Schlossberg, Thresher, 
Ticdeman. 

Not endorsed by l member: Shirley. 

General Comments 

Foster: “It would be very difficult to determine the causes of 
dropouts and transfers, as is suggested in the fifth paragraph of 
the brief. 

“Speaking of the output to the student, (page 53) it may 
be a bit early for some students to have developed some of these 
characteristics and interests. 

“About the needed indexing device mentioned under ‘Si- 
multaneous Operations and Research’: one thinks of the indexes 
now used progressively to narrow the field toward identifying a 
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bird from a book— color, size, shape, habits, and so forth.” 

Pace: “I think we can take some major steps in the direction 
Hersey proposes. But there is no past or present research that can 
honestly tell a student what institution is best for him or that can 
tell a college which applicants, among those who are academi- 
cally qualified, will be ‘best’ for the institution. Moreover, the 
criteria for admission are being changed, and the criteria for suc- 
cess in the institution are also being changed. The best we can 
do, I think, is to describe each to the other (the student and the 
college) in ways both can understand. 

“I can endorse the thrust of Mersey’s brief. But some of the 
particular pieces of information he recommends are neither fea- 
sible to obtain, clearly intcrpretable, nor defensible by current 
knowledge or research. This is why a research program as he 
recommends and as I recommend, needs to run parallel with an 
active program. I fully endorse his final suggestions on simulta- 
neous operations and research.” 

Shirley: “Though I am sure that most of the Commission 
members arc sympathetic to the idea of the Board’s being more 
concerned about furnishing information about colleges and uni- 
versities to the students, Hersey’s proposal is idealistic and un- 
realistic. As a result, I cannot endorse these recommendations. 

“In the first place, I do not feel that the Commission should 
issue demands to the Board; it should consider philosophies and 
policies and make recommendations. The wording of this pro- 
posal is almost in the nature of a demand, since it indicates that 
implementation of this program should be immediate, preceding 
study, analysis, or experimentation. Undoubtedly a program 
such as Hersey suggests should be available. But to implement 
it in the present state of the information-retrieval art would re- 
quire a massive national effort that the priorities of our times 
would not endorse. 

“Universities have spent literally hundreds of thousands of 
dollars to get a small portion of this information and to put it on 
computers so that it is available for analysis. Hersey’s proposal 
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calls for gathering total information from all the colleges and I 

universities of the nation and for keeping this information to- j 

tally current. It calls for gathering and analyzing total informa- j 

tion on the whole person on literally millions of individual col- | 

lege applicants. It calls for bringing these two awe-inspiring data 
banks together in a complete analysis. Compared to this task, 
the federal census or the national tax collection is child’s play— 
not counting the fact that computers are not yet uniform enough 
nor are memory banks massive enough to approach this task. 

Hersey’s idea is a good one, but the Board could not finance it 
nor could the resources of higher education implement it. 

“To endorse an impossible proposal is not wise. So I can 
sympathize, but I cannot recommend, this idealistic suggestion.” 

Thresher: “The objectives sought by this proposal are good, 
and I favor its adoption, but the Commission members need to 
retain a sense of the boundary conditions and limiting factors 
that would condition the effectiveness of any such plan. We need ] 

to ask ourselves how far it would cure what ails higher educa- 
tion. 

“The ‘symmetry’ principle aims at redressing an unbalance 
in a system of reciprocal choice. Thus the proposal would best 
serve the small, affluent fraction of college applicants whose ac- 
cess to the national market is unhindered by high tuition or by 
travel cost. Among colleges it would best serve the small, afflu- 
ent group of institutions who make a big thing of selectivity. 

Another and larger segment of students (say the 80 percent or 
more who attend college in their home state) have only a re- 
stricted choice in any case. For them the plan would be useful, 
but to a lesser degree. For the large and growing number (per- 
haps soon to be a majority) who go first to their local community 1 

college, choice scarcely exists. Their problem is not choice, but 
an improvement of education. 

“The proportion of students in public institutions will con- 
tinue to grow. For these, the selection of a congenial or stimu- 
lating environment will often take the form of a choice among 
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schools or programs in a large, nearby institution rather than 
choosing a college as it is known now. 

“The proposal may tend to implement the extreme, elitist 
views of many faculty members as these search the lists for 
young disciples who will become copies of themselves. It does 
not directly encourage the strongest institutions in one of their 
major, neglected obligations: to demonstrate their ability to 
reach and stimulate a representative cross section of intelligent 
youngsters, rather than giving themselves all the breaks in ad- 
vance by superselection. If a basketball coach accepts only seven- 
footers, you can’t really tell whether he is a very good coach 
or not. I cannot conceive that even had such a plan been in suc- 
cessful operation during the past five years, the basic situation 
would have been greatly changed, in which students are wild 
and colleges are desperate. 

“The breakdown of elitist admissions policies is already well 
under way, with various special arrangements for culturally de- 
prived minorities. Discipline-oriented scholars are encountering 
the world in large, interdisciplinary chunks (or chunks for which 
no discipline has been invented). This trend will have profound 
effects on the prim, compartmented world of academic man. 
There are the major areas that need reform: admissions policies 
that take account of the learning process to be more problem- 
and project-oriented. But college choice will continue to play a 
sufficiently important part to justify the proposal as presented.” 

Abelson: “This brief, as well as the brief by Schlossberg, Hoy, 
and Gordon (‘Regional Centers for Guidance in Continuing 
Education’), appears to me to include laudable objectives. How- 
ever, when looked at in total, implementation of all these pro- 
posals by the College Board does not seem feasible. The costs 
would be too great. If these recommendations are to amount to 
more than rhetoric, further steps would be necessary. Either 
there must be a proposal for a large-scale federal subsidy or the 
Board should arrange a scale of priorities. In view of emerging 
patterns of government interference with education, I would be 
against proposing federal subsidy.” 
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Educational Civil Fights 
and the College Board 

John C. Hoy 



The promise of equal educational opportunity for all students 
remains a distant though compelling myth of contemporary life. 
The College Board, although peripherally involved in several 
important areas of the educational civil rights movement, has 
yet to put the full capacity of its resources, prestige, and organi- 
zational power to work in assisting member institutions in re- 
solving the burning issues related to this struggle. 

As an organization the Board is only beginning to break a 
pattern of token integration. Until such time as the composition 
of the staff, membership committees, and Board of Trustees re- 
flects the ethnic and racial composition of the American educa- 
tion community the College Board will be handicapped in 
confronting the problem. Governance, as well as program and 
policy development, of the College Board will emerge more 
fully as such constituents are included in discussions of the pro- 
found issues faced not only by the Board and its member institu- 
tions but also by that broader student population the organiza- 
tion hopes to serve in the future. 

It is therefore recommended that the College Board take the 
following preferential steps: 

1. Increase minority-group representation on the College 
Board staff to a level that more fully reflects the society at large 
and the commitments already shown by member institutions. 

2. Change the composition of committee membership to re- 
flect the same pattern. 

3. Seek to shift the membership of the Board of Trustees in a 
similar direction. 

Action on all these recommendations should come with a 
minimum of delay, and improvement in one area should not 
excuse lack of improvement in another. As the College Board 
moves to increase the level of minority-group representation it 
should also carefully attempt to use the insight and capabilities 
of minority-group staff to the fullest extent possible. To date the 
Board has not done this. 

In addition, a review of the present financial commitments 
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of the Board as well as a complete analysis of available supple- 
mentary funds in the area of equal educational opportunity pro- 
grams should be developed, with a view toward a dramatic ex- 
tension of activity in this area. 

In essence, the recommendation is for a quantum jump in 
College Board personnel, structure, and finance that will permit 
the organization to face internally as well as externally the most 
serious domestic problem facing the United States. 

Programs that fall short of a major change in the basic nature 
of the Board will undoubtedly fail to engage the organization in 
the level of activity and commitment that is not only required 
but morally just if the practice of the College Board is to become 
consistent with its professed goals. 

At a time when the complexity, the pressures, and the ten- 
sions related to the educational civil rights movement have pro- 
foundly affected member institutions, College Board leadership 
may be able to reassert a measure of perspective and vision with- 
in the education community if it is able to move closer to a struc- 
ture that is representative of the whole society. 

Recommendations 

It is recommended that the College Entrance Examination Board: 

1. Increase minority-group representation on the College 
Board staff to a level that more fully reflects the society at large 
and the commitments already shown by member institutions. 

2. Change the composition of committee membership to re- 
flect the same pattern. 

3. Seek to shift the membership of the Board of Trustees in a 
similar direction. 

4. Review its present financial commitments and completely 
analyze available supplementary funds in the area of equal edu- 
cational opportunity programs, with a view toward a dramatic 
extension of activity. 
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Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
Educational Civil Rights and the College Board 



It is recommended that the College Entrance Examination Board: 

1. Increase minority-group representation on the College Board 
staff to a level that more fully reflects the society at large and the 
commitments already shown by member institutions. 

Endorsed by 14 members : Blocker, Carroll, Coleman, Friedcnberg, 
Gordon, Hcrscy, Holtzman, Hoy, Marland, Pace, Pearson, 
Schlossberg, Thresher, Ticdeman. 

3 members abstained: Abelson, Cross, Foster, ^entice, Shirley. 

2. Change the composition of committee membership to reflect 
the same pattern. 

Endorsed by 14 members: Blocker, Carroll, Coleman, Friedenberg, 
Gordon, Hersey, Holtzman, Hoy, Marland, Pace, Pearson, 
Schlossberg, Thresher, Ticdeman. 

3 members abstained : Abelson, Cross, Foster, Prentice, Shirley. 

3. Seek to shift the membership of the Board of Trustees in a 
similar direction. 

Endorsed by 14 members: Blocker, Carroll, Coleman, Friedcnberg, 
Gordon, Hersey, Holtzman, Hoy, Marland, Pace, Pearson, 
Schlossberg, Thresher, Tiedeman. 

3 members abstained: Abelson, Cross, Foster, Prentice, Shirley. 

4. Review its present financial commitments and completely 
analyze available supplementary funds in the area of equal edu- 
cational opportunity programs, with a view toward a dramatic 
extension of activity. 
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Endorsed by 13 members: Blocker, Carroll, Coleman, Friedenberg, 
Gordon, Horsey, Holtzman, Hoy, Marland, Pace, Pearson, Pren- 
tice, Schlossberg, Thresher, Tiedeman. 

4 members abstained: Abelson, Cross, Foster, Shirley. 

General Comments 

Carroll: “Worthy goals, these, but I would hope they would not 
be achieved in a ‘mechanical' way. I doubt that the Board should 
attempt to make a ‘quantum jump’ in the representation of mi- 
nority groups (one should remember that there are many minor- 
ity groups and subdivisions thereof) on its staff or committees. 
The minority group personnel must be qualified and effective— it 
is difficult enough, we all know, to obtain qualified and effective 
members even from ‘majority' groups. For example, it would 
probably not be too difficult to identify qualified minority-group 
people to serve on certain committees." 

Cross: “I do not think this is an appropriate area for Com- 
mission involvement. It strikes me as rather gratuitous. I have no 
reason to believe that the Board needs the Commission's free ex- 
hortations in this area." 

Hersey: “A minor question of fact: In the first sentence, I 
don't believe that ‘the promise of equal educational opportunity' 
is a ‘myth.' The promise exists. It is uttered often. That it is not 
being fulfilled rapidly enough is evident, but that doesn't make 
it a myth." 

Holtzman: “I agree strongly in principle and hope that the 
recommendations can be implemented. My only reservation 
concerns the possibility that the recommendations might be em- 
braced so literally that the quality of participation and leadership 
in the College Board and its committees might be seriously im- 
paired. Unfortunately there aren't large numbers of minority- 
group members on our college faculties, and those who are tend 
to be overworked by incessant demands on their time and en- 
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ergy to be ‘representatives' on committees, and so on. I am op- 
posed to merely filling slots to dress up the looks of things in the 
name of proportional representation. Obviously we have a long 
way to go, however, before this becomes a problem!” 

Pace: ”1 assume that Hoy knows his facts, but I’d like to be 
assured. For I should suppose that changes in these directions 
have been discussed in the College Board and that progress has 
been made. I’m against sin, but I’m also against criticizing a re- 
deemed sinner. This is why l feel some reservation about en- 
dorsing the recommendations. It may be that the brief is gratui- 
tous; if so, I would rather commend the Board for its actions 
than berate it for its past.” 

Prentice: “In regard to the first three recommendations, I do 
not know how fully the College Board’s current staff ‘reflects 
the society at large’ nor whether there are indeed member insti- 
tutions that have made more effective commitments than the 
Board. 

“There is an element of quicksand in this sweeping proposal. 
Who represents minority groups? Is every black like every other? 
Is a Mexican-American necessarily the best choice for a given 
position merely because other Mexican-Americans have not 
been chosen? 

“The emphasis, it seems to me, must be on removing every 
trace of those prejudices that may have prevented the Board 
from employing the most useful or most highly qualified people 
on account of their membership in some minority group. We 
must not substitute new prejudices that in their turn base hiring 
practices on the candidate’s parentage rather than on his qualifi- 
cations.” 

Shirley: “The Board has given (and is giving) leadership in 
the matter of racial equality.” 
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A Proposal for a Comprehensive System 
of Testing for Job Entry 

Sidney P. Marland 



The College Entrance Examination Board, without necessarily 
meaning to, has become a universal symbol of academic quality 
and prestige in the eyes of high school students, parents, and 
teachers. To be preoccupied with taking the College Board’s 
tests is viewed as important, worthy, desirable, and commend- 
able. Not to be so preoccupied is viewed as something less im- 
portant, less worthy. 

One may readily observe that this is as it should be. Amer- 
ican society is strongly committed to the development of the in- 
tellect. Historically the College Entrance Examination Board 
has served higher education in helping to identify, select, and 
develop the most academically promising young people. Allow- 
ing for all the gaps in the identification process and all the ad- 
mitted limitations of testing, the fact remains that excellence and 
all that this tenn loosely implies in the education system ulti- 
mately finds one of its major yardsticks in the College Board’s 
tests. In at least some parts of the country a student is viewed by 
his school, his peers, his parents, and social institutions, first, ac- 
cording to whether or not he even took the tests; then, of course, 
if he enjoys this status lie is viewed according to his scores. 

The yardstick is not confined to students alone. Ft is not un- 
common for school systems to examine themselves, particularly 
their high school faculties, in relation to the median scores at- 
tained by a given class, and for high schools wirhin a city to 
compare notes (perhaps secretly) on their “success” with high 
scoring students or class averages. The high school that has not 
entered very many contenders, or perhaps shuns comparing 
scores earned with colleagues across town, is not likely to be re- 
garded with respect within the system. 

That the Scholastic Aptitude Test (sat) and the Achievement 
Tests are symbols of excellence is a fact. Psychologists and col- 
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lege admissions officers may deplore this condition as unsound, 
exaggerated, invalid, and unintended, but it is a fact. Altogether, 
it is probably a good thing, giving some system and standard- 
ization to the vagaries of a pluralistic school system that pos- 
sesses few other constants. The great virtue of the College 
Board is its systematic design for facilitating entrance into vary- 
ing colleges by students of widely varying school and geo- 
graphic backgrounds. It is this important virtue that suggests the 
need for developing a corresponding arrangement for systemat- 
ically matching high school students with a variety of jobs. 

A very large proportion of young people arc excluded from 
the select society that the Board has unintentionally constructed. 
If it is estimated that in the near future as many as half of the 
young people will come under the benign mantle of the College 
Board, and other similar testing services, the other half will not. 
Notwithstanding the swift increase in numbers of youth taking 
the examinations, and the corresponding increase in numbers 
engaging in postsccondary education, a ball-park guess would 
suggest that given the full age-grade population in the public 
secondary schools (retaining the dropouts) fully half of the 
young people will not attend postsccondary school in the fore- 
seeable future. At least some of them will undertake a period of 
full-time work in some form before returning to education. The 
vast majority of these 50 percent do not, at high school age, see 
themselves as likely candidates for higher education. 

Indeed, this “other half" to a large extent do not sec them- 
selves very clearly at all. Some of them may have undertaken 
the “business curriculum" (by whatever name), and some have 
had other kinds of vocational training, but the majority are en- 
gaged in that educational euphemism called the general curricu- 
lum. This means that the student will graduate from high school 
without intentions for more education and without a salable 
skill. There is no longer a productive place in our society for 
someone who has this kind of limited educational experience. 
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While his individuality may have been enriched by his “general 
education,” he is without the means to apply his energies to a 
productive task. 

There is an unfortunate and silent history underlying this 
condition. In its simplest form it is the involuntary or voluntary 
expression of disdain by a significant proportion of American 
people for those whose work is performing services or tasks that 
involve manual or manipulative activity. Endemic snobbery has 
tended to classify the mcnual-manipulative task too simply as 
nonintellectual, noncreative. While our public schools over the 
years should have been enrolling well over 60 percent of the 
high school students in vocational-technical programs (the rest 
of the students planning for higher education), typically only 5 
to 10 percent have been engaged formally in skills training. The 
remainder have been caught up in a modified college prepara- 
tory curriculum, considered somehow more respectable than 
vocational education, even though college preparation was not 
intended, either by the school system or the student. But the 
academic program, culminating in the College Board’s sat for 
some, has been the singular symbol of capability for entering the 
“managerial” class or the society of intellectually elite. No other 
common symbol of a favored social stratification has been avail- 
able for those whose interests and talents lie in other directions; 
excellence by definition has resided in the college preparatory 
program. The noncollege preparatory student is therefore rele- 
gated to something less than excellent. 

It is here proposed that an institution be created that will 
give to vocational-technical study the same level of respect and 
prestige that the liberal arts studies now have, and will recognize 
excellence in areas that are not primarily intellectual. This is not 
an easy task in a society whose values have so brightly illum- 
inated the virtues of higher education. This proposal is not in- 
tended to diminish the importance of the present academic pro- 
gram. On the contrary, it seeks to elevate the arts of the world 
of work to the level of the liberal arts as socially desirable goals 
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for students. It is emphasized that a student is not obliged to fol- 
low one ‘‘track” or the other, as there are no fixed tracks. Most 
likely he will find his fulfillment in a combination of academic 
and vocational-technical studies, whether his goal is college or a 
job. 

The proposed structure for the examination of those young 
people directly entering the world of ^ork (to which they 
would bring salable skills as well as inteli^Ltial competence) 
would serve at least as many individuals and probably more than 
will be served by the College Board in its present form. The na- 
tional trend toward the comprehensive high school points 
clearly to the objectives of insuring that every person complete 
high school and then either enter higher education or use the 
salable skill he learned there. This leads to the corollary objective 
that calls for immediate and appropriate employment of all high 
school graduates not entering higher education, homemaking, 
or the armed forces. 

The high level of mutual interest and cooperation between 
colleges and secondary schools has effectively supported the 
work of the College Board. The greatly broadened testing plan 
proposed would call for a correspondingly high level of mutual 
interest and cooperation between the secondary schools and the 
business and labor communities. This is a more complex and 
challenging interrelationship than the one between schools and 
colleges; however, there is strong evidence to suggest a readiness 
in business and labor to work much more closely with the sec- 
ondary schools. 

The proposal assumes that the idea of a comprehensive high 
school will grow and be realized eventually. The high school 
curriculum must offer a wide range of vocational and technical 
programs, including the appropriate general studies necessary 
for adult citizenship and vocational competence, along with the 
college preparatory program. Roughly half the comprehensive 
high school’s students would be enrolled in the vocational and 
technical programs with appropriate general and academic 
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courses, and half in college preparatory programs. Many of those 
entering higher education will enroll in two-year community 
colleges or technical schools. 

The articulation of the curriculum between the technical 
program of the high schools and the postsecondary technical in- 
stitutions becomes highly important. For example, a young man 
engaged in studying computer programing in high school (or a 
young woman interested in practical nursing) should proceed, if 
he wishes, to planned levels of increased curricular sophistication 
in his postsecondary training. The comprehensive examination 
in vocational and technical study has relevance to this transition. 
The technical institution, or the employer, and the student 
would have information for admission and counseling purposes, 
just as colleges and students have such information through the 
College Board’s tests. 

The basic theme underlying this proposition is that all young 
people possess potentials for excellence in one form or another. 
The historic bias favoring intellectual excellence, as revealed in 
contemporary tests such as the sat, has tended to foreclose from 
systematic recognition the young person whose talents lie in 
other directions. Society and the teaching profession have given 
high theoretical prominence to individuality in schooling. Yet 
the reality of the College Board introduces a prejudice that im- 
plies a repudiation of those individual personal qualities, other 
than intellect, that the learner possesses. Therefore, since voca- 
tional-technical education comprises, or should comprise, the 
learning emphasis for fully half the nation’s young people, the 
College Board is urged to broaden its spectrum of competencies 
tested to include personal qualities of career readiness not now 
included in the sat. The unintended “tracking” imposed by the 
sat would be stopped. Indeed, the comprehensive career-entry 
testing design here proposed would serve to obliterate the impli- 
cations of high and low tracks. 

At no time would a student be classified as exclusively col- 
lege preparatory or vocational preparatory. He could always 



choose to concentrate his studies in one directbn or the other. 
Indeed, he could, with good precedent, enter college after two 
or three years of job performance, with increased maturity and 
resources. 

The labor force in the United States now has about 55 per- 
cent of the people in service jobs as distinct from production 
jobs. Prior to i960, more than half the labor force were involved 
in the production of goods rather than services. It is probable 
that by 1980, 65 to 75 percent of all working people will be en- 
gaged in service occupations. Also, many new occupations in the 
semiprofessional, paraprofcssional, and technical levels have 
emerged since World War II. Competent employees for such 
jobs are sorely needed and remain in short supply. These occu- 
pations do not require a baccalaureate degree, but in most in- 
stances they do require secondary education or some post-high 
school education and training. 

Examination of the labor force reveals a nonsymmetrical dia- 
mond configuratbn. The professbns and high-level managerial, 
scientific, and technical positions occupy the top of the diamond; 
the overwhelming majority of the labor force lies in the middle 
of the diamond; and the bottom of the figure includes the rapidly 
declining number of unskilled and semiskilled jobs in our econ- 
omy. 

Adequate training for entering a job on the semiprofessional, 
paraprofessional, and technical levels requires secondary school 
or postsecondary school education of from about 1 year to 2V2 
years. It is important to note that formal training in an institution 
is essentially designed for competence required in an entry job. It 
must be assumed that virtually all workers above the totally un- 
skilled level will receive additbnal training and periodic retrain- 
ing made available by the employer in the plant or through an 
arrangement with a public or private educational institution. 

The proposed testing institution would establish a valid 
measure of competence for job entry, whether on the completion 
of high school, or of postsecondary technical education, or both. 
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At present such job-entry testing is at best unorganized, and at 
worst nonexistent. Standardized tests are available in some fields 
of vocational and technical education; they may or may not be 
used by individual teachers or school systems. Entry into union- 
managed apprenticed trades generally requires at least perform- 
ance testing, and there is apparently a tendency to “screen out” 
rather than to “guide in.” Business and industry have varied 
types of job-entry examinations, administered internally or ex- 
ternally by employment offices. It is the intent of this proposal to 
bring consistency and order to the job-entry system for all fields 
of work that formal vocational-technical education offers now 
and in the future. 

There is a great need for providing useful measures for the 
K- 12 or K-14 span of experience for students whose competen- 
cies lie in directions other than academic. It would appear that 
the College Board, or a new organization, should immediately 
go beyond the sat to provide measures of salable skills. While 
this brief does not go beyond this aspect of recognizing indivi- 
dual excellence other than academic, the discussion can be ex- 
tended still further. For example, high school students have other 
categories of individual excellence that do not fall into either the 
sat context or the salable skills context. Demonstrated compe- 
tence in music or art, deep commitment and sensitivity to social 
responsibility, political and social leadership, athletic skill in 
many aspects that suggest adult careers or avocations— these arc 
qualities of individual excellence that are ignored by the present 
culminating testing scheme. It is possible that the explorations 
proposed here in support of extending the culminating test to in- 
clude vocational skills, may be still further extended by the ex- 
periment to include these more abstract qualities. 

In thinking about greatly increasing the number of students 
tested, as well as increasing comprehensiveness of the content of 
the tests, great care must be taken to avoid redundancy and to 
insure the privacy of all students within the testing structure. As 
the proposed comprehensive system of testing takes effect, exist- 






ing tests with similar scope or purpose should be eliminated. In- 
deed, some progress is already being made in this direction. 

As to privacy, the proposed system of comprehensive testing 
would still rest on the professional integrity of responsible staff 
members, both in schools and in the field. The field will include, 
however, not only college admissions officers, as now, but also 
the whole of the employment world. Obviously the significant 
data describing a student must be transmitted systematically to 
the object agency (employer). The data must be honest and ob- 
jective, just like the essential descriptions that now come from 
the secondary schools on college admissions application forms. 
The degree of privacy hinges largely on the integrity of the in- 
dividuals compiling and receiving such data, whether the object 
is college entry or job entry. The significance this proposal has 
for a student’s privacy lies in the fact that lay citizens share the 
job of appraising him for job entry. However, as lay members of 
boards of education are responsibly privy to virtually any infor- 
mation surrounding any individual in a school system, it seems 



reasonable to assume that people of corresponding integrity will 1 

be able to accept the responsibility of maintaining the student’s -jj 

privacy. ] 

A basic concern related to the proposal of an organization 
for testing students’ competence for job entry is the fact that } 

there is doubt that vocational-technical education belongs in j 

high school. Some people declare that the public schools should 



confine themselves to academic teaching and that postsecondary J 

institutions or on-the-job training in business and industry '■ 

should satisfy vocational-technical needs. There are also those ;■] 

who, viewing the swiftly changing technology in business and 
industry, hold that it is not efficient to train a young person in a 
given field because that field will change by the time he enters | 

the work force. Such observations do not take account of the J 

reality of high schools today. This proposal is aimed toward j 

making the schools relevant to those young people not planning j 

to attend college and suggesting the feasibility, even the attrac- j 



O 




Sidney P. Marian d 



tiveness, of some plan other than immediate entrance into a typi- 
cal college career. It is true that much learning will follow gradu- 
ation in on-the-job situations. It is also true that the techniques 
of job performance are changing swiftly. Some competent ob- 
servers declare that a worker today may have to change his work 
techniques as many as six or seven times in his working lifetime. 
But in this proposal it is held that the young person in school has 
to start to equip himself for job entry, acknowledging the 
changes that lie ahead. Part of his learning would be the very 
comprehension of the reality of change. Business and industry 
should not be expected, as they now often are, to provide re- 
medial education in basic academic skills, nor to create a school 
for typists or welders or draftsmen or electricians. The student 
must have a starting point, with measurable competence in a 
salable skill, so that he can enter the stream of in-service growth 
and changing technology at whatever level his competencies 
suggest. 

While the fundamental job-entry skills of the vocational- 
technical program are important, an equally important part of 
the high school program should be concerned with attitudes and 
personal qualities. The young person being educated for job en- 
try must have extensive counseling, encouragement, and in most 
cases, successful field experience to help him toward being a 
well-developed person. The very fact that he has a job-entry 
skill has much to do with his psychological well-being. 

For many young people, especially those from deprived cir- 
cumstances in big cities, the schools have not been meaningful. 
Continuing exposure to a general curriculum, without tangible 
relationships to future employment, is understandably rejected. 
The high proportion of dropouts, ranging as high as 60 to 70 
percent in some city schools, is unquestionably a product of 
curriculum irrelevance, at least in part. Further, the ascending 
tempo of student unrest, stridency, and hostility can be attri- 
buted to this same irrelevance. The schools are struggling to cor- 
rect this condition. Their struggle would be significantly assisted 
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by the prestige attached to the proposed system of assessing vo- 
cational-technical education. The built-in goals, the new tangi- 
ble sources of motivation, and the enlarged dignity of skill- 
centered programs envisioned in the universal testing system 
could go far to redress the dangerously low status afforded many 
secondary schools, especially in the cities. 

It should be emphasized that in this recommendation a sub- 
stantial change in secondary school curriculum is seen for about 
half the student population. As a larger proportion of the coun- 
try’s youth finish high school and more enter higher education 
and skill-centered employment, the curriculum must be reorga- 
nized to fit current realities rather than reflecting to so large an 
extent the traditions of the liberal arts experience for a profes- 
sional-managerial and intellectual elite. Not only must basic con- 
tent be modified, but there must be a reorganization of the rela- 
tionships of academic disciplines to the newly established clusters 
of vocational-technical offerings. Lively urban school systems 
need to be engaged in the curricular evolution that comes with 
developing a comprehensive high school. The proposed testing 
system must reflect the changed curriculum. 

Since American colleges and universities do in fact substan- 
tially influence the curriculum in the elementary and secondary 
schools, the College Entrance Examination Board may quite 
properly undertake a share of the responsibility for the reorienta- 
tion of the schools so that they will more effectively serve the 
needs of all students. Thoughtful Americans are not likely to ac- 
cept a theory of change in secondary school curriculum that has 
merely trickled down from leisurely reforms in higher educa- 
tion. The problems of the urban population have already reached 
the crisis stage, and the College Board has an opportunity to 
share in the search for solutions. The Board might well assume 
some initiative in restructuring secondary education through 
the processes proposed here. If indeed the College Board has 
been. a highly influential party to the curriculum patterns of the 
academic program, as I believe it has, then this same influence 
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could well be brought to bear in the design of the vocational- 
technical programs aim?d especially toward urban educational 
issues. Such an extension of the Board’s function would suggest 
either a change in name or the creation of a counterpart organi- 
zation. 

There is much to be said for having all students participate in 
a single general measurement process at the end of high school. 
Such a measurement process would have certain common char- 
acteristics for all students and special elements related to voca- 
tional or college goals that would lead in many different direc- 
tions. All parts would be equally useful, worthy, and respectable; 
no part would be viewed , as possessing more prestige than an- 
other. 

Accordingly, it is proposed that the College Board undertake 
or sponsor a major study of the proposition expressed in this 
brief design a model, and conduct a pilot exercise to weigh the 
worth of the enterprise. The details of how to construct this 
proposed testing institution for vocational-technical competence 
require careful study. This proposal only states the problem and 
calls attention to the opportunity for a major breakthrough in 
present school organization and employment practices. The cen- 
tral message of the proposal is only partly the design of a system 
of mechanical arrangements for job-entry assessment. The most 
significant message of the proposal is that the abstract qualities 
of prestige, dignity, and worth associated with the vocational- 
technical tests and with the agency that sponsors and administers 
them would, in time, affect the vocational program as the pres- 
ent College Board tests now affect the academic program. Per- 
haps nearly all secondary school graduates will someday take a 
new form of College Board examination without sensing an 
implied value in one kind of post-high school plan as against 
another. It would be naive to suggest that all “managerial pres- 
tige” would be removed by the processes proposed here, but it is 
likely that the present disparity between the college preparatory 
and vocation preparatory programs can be sharply diminished. 
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Once this is achieved, there should be a change in the name of 
the College Entrance Examination Board, perhaps to Career 
Entry Examination Board or Continuing Education Examining 
Board. It would, as a single institution, carry out the present 
functions of the College Board and the new functions suggested 
in this proposal. 

The present College Board should be used as a model for 
drawing up the new organization, keeping in mind certain facts 
and goals. 

a. There should be a thorough assessment of the present vo- 
cational testing materials now available from all sources. Where 
these arc found valid they should be used; in fields in which they 
arc inadequate or nonexistent new testing materials should be 
developed. 

b. The battery of tests should include measures of the stu- 
dent’s achievement in basic academic subjects pertinent to the 
needs of the occupation for which the student is being assessed. 
These measures should probably be quite different from those 
used for predicting college success. 

c. The design for the test program should include provisions 
for reporting the student’s personal characteristics, including at- 
titudes, self-discipline, responsibility, relations with peers, and 
so forth. It may include objective measurement instruments, ap- 
praisal by teachers and counselors, or both. Problems of student 
privacy should be considered. 

d. A performance component of the test might include an 
arrangement for the establishment of a panel of lay evaluators in 
collaboration with appropriate professional staff members of the 
school. Each comprehensive high school might have several such 
panels regularly appointed, according to the general occupa- 
tional category— that is, health sciences, construction trades, 
electronics, and so forth. The panel would include expert prac- 
titioners as well as ordinary members. The candidate would be 
judged by the panel on his demonstrated competence in the skill 
he has learned, and he would be formally certified for a partic- 
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ular level of job entry accordingly. Such a panel would be com- 
parable to a board of education in service and responsibility, 

c. The present effect of testing is seen by many young people 
as a “gate-keeping process,” excluding rather than including 
many individuals. The spirit of the proposed system should em- 
phasize affirmation, access, and supportive counsel. Accepting 
job placement as one of the goals of the system, it should be as- 
sumed that there is an appropriate job for every student who 
completes high school and looks for employment or further 
technical education. The testing design proposed should serve as 
the “gate-opener” to all young people, giving access at the entry 
level to which rhe individual’s skills and interests arc appro- 
priate. 

Following the practice of the present College Board, the 
new organization would give a student a score or rating express- 
ing the level of competence he has achieved in relation to the 
entry-level criteria for the field of work he wants to enter. His 
score or certificate of performance would become his credential 
wherever he applies for a job. This, again, calls for the lively en- 
gagement of the business and labor communities in adapting 
their job-entry requirements to the new credential system. 
Superficially, at least, it would seem likely that the new system, 
when compared with current practice, would be welcomed 
cordially by employers. 

Other possible indirect outcomes of this proposal should be 
viewed with interest. The presumed new battery of tests can be 
used not only for those who are consciously vocationally or 
technically oriented, but also for some college-bound youth. Its 
existence and effect might encourage some young people who 
would go to college for lack of any better alternative to consider 
nonacademic paths in a new light, influenced by the guidance 
program. When a student secs that a bricklayer can earn $18 an 
hour, own a boat and two or more cars, and vacation in Florida, 
a white collar job may not seem so attractive. If bricklayers or 
automobile mechanics or practical nurses were more highly re- 
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garded in our value system there might be fewer misfits in the 
nation’s colleges. 

The new test battery might also be used for two other guid- 
ance purposes. In an era of increased leisure time, shortages of 
skilled workers, and high prices, it may be more important to 
have an avocation— perhaps one that involves skilled work such 
as carpentry or electrical work. The nonin tcllcctual and the intel- 
lectual person can derive much satisfaction from such pursuits. 
The new tests might point the way to an aptitude for an enjoy- 
able pastime, much as an exploratory college course stimulates a 
student’s interest in the scholarly arts or sciences. 

Another use of the tests for guidance would be in on-the-job 
reeducation and adult education in industry. A new set of indi- 
cators of aptitude and entry classification should be welcome, 
and experienced employees could move on to better jobs 
through the systematic resources of the testing process. The re- 
sults of such tests, furthermore, might induce employers to ac- 
cept and train unskilled persons whom otherwise they would 
reject. 

The suggested system of assessment measures is illustrative 
only, and it needs further scrutiny and extensive development. 
No attempt is made to identify what agency might undertake 
this program if the College Entrance Examination Board should 
find it inappropriate. There are, obviously, existing institutions 
that could bring experience and wisdom to the task. On the 
other hand, the extraordinary dimensions of this proposed pro- 
gram may call for the construction of a new agency. It is urged, 
however, that the Board undertake at least pilot programs, 
merging its present academic function with the broader cover- 
age, while still, of course, maintaining the academic functions. 

In any case, a major study of the issues and assumptions of- 
fered in this proposal appears to be warranted. Such a study, 
gathering the counsel and creative energies of scholars and prac- 
titioners should be undertaken, under the auspices of the College 
Board, the United States Office of Education, or perhaps both. 
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In viewing the idea of tests for vocational and technical job 
entry, it must be remembered that the transcending goal of the 
system is the enhancement of the learning environment for the 
very large number of young people, especially in big cities, who 
are untouched or rejected by the college-oriented traditions of 
secondary education. The goal of preparing well-developed 
people for entry to the world of work is paramount. The self- 
respect that should derive from the newly established impor- 
tance of the vocational area of education is a basic ingredient of 
the design. Indeed, the presence of a universal testing arrange- 
ment and the process of its application may be more important 
influences on the behavior of young people than the ultimate 
product of job-entry machinery. 

Recommendations 

It is recommended that the College Entrance Examination Board; 

(1) Take steps to increase greatly the scope of its testing pro- 
gram to include the measurement of student competencies other 
than those reflected in measures of academic promise for college 
entrance. 

(2) Undertake initially a major study, including a pilot pro- 
gram to develop appropriate materials and procedures, of the 
feasibility of assessing qualifications and classifications for job 
entry and for postsecondary technical training. 

(3) If the results of the study and the experience with the 
pilot program are promising, consider expanding its function to 
serve all high school graduates, including those entering the 
work force directly as well as those planning to go to college, 
and changing its name accordingly to “Career Entry Examina- 
tion Board,” or such other appropriate name as would accu- 
rately reflect its expanded clientele. 



Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
A Proposal for a Comprehensive System of Testing 
for Job Entry 

It is recommended that the College Entrance Examination Board: 

1. Take steps to increase greatly the scope of its testing program 
to include the measurement of student competencies other than 
those reflected in measures of academic promise for college en- 
trance. 

Endorsed by 15 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Fricdenberg, Gordon, Holtzman, Floy, Marland, Pace, 
Prentice, Shirley, Thresher, Tiedeman. 

Not endorsed by 3 members: Hersey, Pearson, Schlossberg. 

2. Undertake initially a major study, including a pilot program 
to develop appropriate materials and procedures, of the feasi- 
bility of assessing qualifications and classifications for job entry 
and for postsecondary technical training. 

Endorsed by 13 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Holtzman, Hoy, Marland, Pace, 
Prentice, Shirley, Thresher, Tiedeman. 

Not endorsed by 3 members: Hersey, Pearson, Schlossberg. 

3. If the results of the study and the experience with the pilot 
program are promising, consider expanding its function to serve 
all high school graduates, including those entering the work 
force directly as well as those planning to go to college, and 
changing its name accordingly to “Career Entry Examination 
Program,” or such other appropriate name as would accurately 
reflect its expanded clientele. 
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Endorsed by 14 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Holtzman, Hoy, Marland, Pace, 
Prentice, Thresher, Tiedeman. 

Not endorsed by 4 members: Hersey, Pearson, Schlossberg, Shirley. 
General Comments 

Abelson: “1 wholeheartedly endorse Marland’s brief, which calls 
attention to very important deficiencies in current procedures. 
Implementation of the remedies he suggests should have a high 
priority.” 

Carroll: “I endorse these recommendations only insofar as 
they might lead to a pilot study to test their validity. I feel that 
there may be many obstacles— financial, social, and educational 
—to the implementation of these proposals. In certain areas the 
College Board would be competing with industry, labor unions, 
or professional organizations in the testing of candidates, if these 
proposals were carried out to the letter. Nevertheless, the pro- 
posals are in the right direction, and I feel that at least they de- 
serve serious consideration/’ 

Hersey: “I endorse without reservation Marland’s thesis and 
his statement of the urgent need for the services he describes, but 
I feel that the College Board will have its hands full to effect the 
symmetry we have discussed in its role of facilitating entrance to 
higher education. It would be valuable, of course, to have the 
Commission endorse and support Marland’s thesis and recom- 
mend that others take up the task.” 

Holtzman: “Some elements of this proposal could be com- 
bined with the plan for regional guidance centers in the brief by 
Schlossberg, Hoy, and Gordon. However, in Marland’s brief 
there is too much emphasis on testing, I think.” 

Hoy: “If these recommendations are as far sighted as I be- 
lieve they are, within a decade the College Board may well be 
devoting most of its energy to the assessment and encourage- 
ment of talents not limited to the aptitude measurements and 
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achievement tests of the 1950s and 60s. 

“The Gordon and Marland papers combine to recommend 
a major and constructive departure for College Board programs 
that should have the effect of not only enriching the variety of 
tests but also dramatically broadening the Board’s clientele. 

“The testing programs implied by this paper indicate a con- 
tinuing approach to career and educational planning and devel- 
opment for the individual— not one- or two-shot testing but 
rather appraisal, follow up, and reappraisal.” 

Pace: “I think the brief is an extremely important one. 

On page 69 I am not sure that the point about the College 
Board’s great virtue is a good one, because the Board doesn’t 
really ‘match’ the student with the college; the sat serves to 
string people out along a single dimension. 

“The ball-park guess (page 69) may underestimate the num- 
ber who continue education beyond high school. Are there 
figures that can be cited? But whether there are or aren’t, the 
basic argument is equally valid, for the issue is recognition, re- 
ward, and respect.” 

Pearson : “I’ve been reasonably expansive in thinking about 
new College Board programs, but this proposal seems to me to 
be outside the Board’s mission and capabilities.” 

Prentice : “I strongly sympathize with the aims of this pro- 
posal, but I hope it will be developed conservatively and step by 
step. I can only partially endorse the second and third recom- 
mendations. The testing of readiness for admission to post-high 
school vocational education is a highly appropriate College 
Board development, but I am less sanguine about ‘job entry 
testing,’ which is incredibly diverse and would require untold 
research before it led to suitable social gains.” 

Schlossberg: “I agree completely with Marland’s analysis. My 
disagreement is with his proposed solution. I do not feel that 
wholesale testing will change testing from ‘gate-keeping’ to 
‘gate-opening.’ I am sorry to disagree, but I just can’t support 
the expansion of testing.” 



Shirley: “Recommendation i: Experimentation on a broad 
scale should be a purpose of the Board. And if there is to be any 
symmetry, more than one avenue toward a satisfying life should 
be offered the student. This may well be a function of guidance, 
rather than testing, but certainly the feasibility of testing should 
be explored. 

“Recommendations 2 and 3: It is certainly fitting for the 
Board, in the interest of symmetry, to run such a pilot program 
on job entry. But this should be a research pilot program only. 
If such research shows need for a national effort, and if the re- 
search shows tests to be a feasible way of solving this problem, 
the operations should be taken over by some other national 
agent, since this program goes far beyond the scope or ability of 
the present organization.” 

Thresher: “I support this proposal, at least in principle. The 
scale is so immense that it may well be that we cannot look, at 
present, much beyond a pilot program. Both this brief and that 
of Schlossberg, Hoy, and Gordon are of a character and scope to 
shake the Board out of its somewhat complacent servicing of the 
conventional academic establishment.” 



A Program for Providing Information 
about Colleges to Applicants 

C. Robert Pace 



Granted the virtue of a greater symmetry of information be- 
tween applicant and institution in the process of deciding who 
goes where to college, the College Entrance Examination Board 
will need to develop a program for doing this that is relevant, 
feasible, and impartial. Specifically, what information about the 
institution is relevant? How is relevance determined? How is 
impartiality assured? How is validity judged? And what is feas- 
ible? These questions cannot be wisely answered overnight. 
They require systematic, scholarly, and imaginative considera- 
tion. The consequences of the inventions and answers that are 
arrived at may be as significant as the consequences have been of 
previous inventions of measures of scholastic aptitude and 
achievement and grade-point averages. 

The purpose of this brief is threefold: (i) to recommend that 
the College Board support a research and development program 
along with the earliest possible introduction of a mutual infor- 
mation exchange system between applicants and Board member 
colleges; (2) to suggest something of the content and complexity 
of such a research and development program; and (3) to suggest 
possible forms that an information system might take. 

Estimating the Dimensions of the Problem 

Realistically, I assume that the College Board is concerned with 
providing information about Board member colleges to the ap- 
plicants to those colleges. Idealistically, I assume that the Board 
is concerned with a potentially nationwide program without 
regard to what institutions belong to what organizations— at 
least that would be true if the Board accepted the student as its 
potential client just as it now accepts the college as its actual 
client. 

Accepting the student as a client, however, does not neces- 
sarily mean that the Board would be dealing with several million 
high school graduates in relation to more than 2,000 colleges and 
universities in some mutual information exchange system. The 
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question “Who goes where to college?” can be answered; but 
the question of how much choice is involved in this distribution 
of students to colleges cannot be answered with quite the same 
assurance. For many students no choice (realistically) is open. 
“The opportunity for students to choose where they will go de- 
pends on several conditions: on what is available near at hand, 
on knowledge of what is available elsewhere, and on the stu- 
dents’ talents and financial resources. The combined possession 
of high academic talents, money, and scholarships will enable 
some students to make national choices among the best public 
and private institutions. . . . There are selective students as well 
as selective institutions. . . . For most students going to college, 
however, the choices open to them will be more limited, based 
mainly, as they are now, on the circumstances that the place is 
near and the price is right. . . . Any national effort toward selec- 
tive assignment, which told people where to go and what was 
good for them, would be contrary to our deepest traditions’ of 
freedom and morality. Distinctive institutions and distinctive in- 
dividuals will just have to find one another, as they often do 
now; and they will be more likely to do so as information about 
each becomes more available to both, and as the resources which 
permit mobility are increased. . . .’ M 

The point of these comments is simply that how one defines 
the magnitude of the problem will have a bearing on what kind 
of information about colleges is really relevant and useful for the 
applicants. If the clientele is limited to the students who can 
afford to be nationally selective, then it is important and useful 
to make subtle distinctions between Harvard, Yale, and Prince- 
ton. But if the clientele is larger and more heterogeneous, then 
such distinctions are of minor utility; whereas differences be- 
tween those institutions and, for example, the University of 
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Northern Iowa, Ouachita Baptist College, and Eastern Oregon 
State College are more important. The content of the informa- 
tion system will depend on th.* kind of discrimination the system 
is designed to facilitate. One needs to face this policy issue before 
one can realistically consider the question of what information 
will be useful to the applicants. 

Examining the Information That is Available from 
Open Sources and Published Instruments 

A very substantial literature has accumulated, especially over the 
past dozen years, that has a bearing on the characteristics of col- 
lege environments. Basically, researchers have used one or more 
of five rather different approaches in describing college environ- 
ments. 

i. Colleges have been characterized by the kind of people 
who go to them. California Institute of Technology can be char- 
acterized as a place in which all students have scores of 700 or 
higher on the mathematical sections of the College Board’s 
Scholastic Aptitude Test (sat). The College Handbook , published 
by the Board, provides information of this type— sat scores, high 
school class rank, and so forth of applicants and enrolled fresh- 
men. Bennington can be characterized as a place where 80 per- 
cent of the entering students have traveled abroad, whereas at 
the University of South Dakota perhaps no more than 5 percent 
of the entering students have traveled abroad (this is a guess, not 
a known fact). Educational Testing Service’s College Student 
Questionnaire (csq) is used in several hundred institutions. 
From the csq one can characterize the student body according 
to educational and vocational plans, family independence, peer 
independence, liberalism, social conscience, cultural sophistica- 
tion, and with respect to value orientations described as aca- 
demic, collegiate, vocational, and nonconformist. Another 
widely used test is the Omnibus Personality Inventory (opi). 
From the opi one can characterize student bodies along such di- 



C. Robert Pace 



mensions as thinking introversion, theoretical orientation, esthet- 
icism, complexity, autonomy, religious orientation, social orien- 
tation, impulse expression, personal integration, anxiety level, 
altruism, practical outlook, and masculinity-femininity. The 
Stern Activities Index and the Edwards Personal Preference In- 
ventory describe students, and student bodies, with respect to 
the strength of various personality needs such as needs for under- 
standing, affiliation, dominance, deference, order, and so forth. 
The Allport- Vernon-Lindzey Study of Values assesses the relative 
priority of theoretical, esthetic, social, religious, political, and 
economic values. From John L. Holland’s Vocational Preference 
Inventory, and from other such tests, one can describe student 
bodies with respect to a variety of occupational orientations and 
interests. Conservatively, considering only the most widely used 
measures, I would estimate that they contain about 2,000 more 
or less different items and produce about 100 more or less dif- 
ferent variables or measures along which one could characterize 
and compare the student bodies of different colleges and uni- 
versities. 

2. Colleges have been characterized by their programs. One 
docs not go to Sweet Briar to study engineering nor to South 
Dakota School of Mines to study music. The catalog is the basic 
source for this type of information. But the catalog is not 
enough. What is also relevant to the atmosphere of the institu- 
tion is the proportion of students in different fields. A college in 
which 40 percent of the students arc in engineering is quite dif- 
ferent from one in which 40 percent of the students are in educa- 
tion. 

3. Colleges have been characterized along various census or 
demographic variables. Again, many of these are common and 
familiar— such as size, form of control, for men, women, or 
both, highest degree offered, rural or urban, residential or com- 
muter, and so on. Alexander W. Astin has grouped information 
of this kind into five, general dimensions that he calls affluence, 
size, masculinity, homogeneity of offerings, and technical em- 
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phasis. These overlap somewhat with what 1 have classified as 
program variables. There are others that may be important, such 
as class size, ratio of undergraduate to graduate students, ratio of 
part-time to full-time students, and so on. 

4. Colleges have been classified by how students behave in 
them. The Educational Testing Service (ets) surveys, by Rich- 
ard E. Peterson, of the incidence of organized student protest 
are an example of this approach. Astin, in some of his research 
for the American Council on Education, has been following a 
similar line, asking, for example, how many students report that 
they have had a blind date, have drunk beer, slept in class, 
argued with a professor, and so forth. From factor analysis of 
information of this kind Astin develops indexes for characteriz- 
ing the peer environment, the classroom environment, and the 
administrative environment. 

5. Colleges have been characterized by what the students 
who attend them perceive to be generally true about them. This 
collective perception or consensus approach is typified by the 
College Characteristics Index, the College & University En- 
vironment Scales (cues), by similar scales developed by Donald 
L. Thistlethwaite, by some of the semantic differential scales de- 
veloped by Lawrence A. Pervin, and by similar but much less 
widely used sets of items that other investigators have tried out. 
The College Characteristics Index, originally developed by Pace 
and George G. Stern and now used exclusively by Stern, pro- 
duces measures described as intellectual climate, vocational cli- 
mate, aspiration level, student dignity, self-expression, group 
life, and social form. One of these, intellectual climate, is parallel 
to various intellectual needs measured by Stem’s Activities In- 
dex. The other environmental press scales are not parallel, or at 
best are only slightly parallel, to corresponding needs scales in 
the Activities Index. Pace’s cues, distributed by ets, produces 
characterizations of the environment along dimensions labeled 
scholarship, awareness, community, propriety, and practicality. 
In the new second edition there are additional scales labeled 




91 



C. Robert Pace 



campus morale, and quality of teaching and faculty-student 
relations. Thistlethwaite’s instrument is divided into 12 faculty- 
press scales and 1 1 student-press scales, with titles such as faculty 
press for vocationalism, faculty press for advanced training, fac- 
ulty press for affiliation, faculty press for independent thinking, 
student press for academic achievement, student press for estheti- 
cism, student press for opposition to faculty influence, student 
press for reflectiveness, and soon. Astin has also developed some 
“college image 11 items, which produce factors labeled academic 
competitiveness, concern for the individual student, school spirit, 
permissiveness, snobbishness, emphasis on esthetics, flexibility of 
the curriculum, and emphasis on social life. Pervin’s semantic 
differential scales include 52 concepts, which can be collapsed 
into 13 factors having such labels as impulsivity-inhibition, lib- 
eral idealism-conservative pragmatism, optimism-alienation, 
and cosmopolitan-provincial. Altogether, in the most widely 
used consensus or perceptual measures there are probably about 
1,000 more or less different items, which in various combinations 
produce some 20 to 50 more or less different scales or variables. 

In a nationwide study of college students, alumni, and en- 
vironments that we are currently conducting at the University 
of California, Los Angeles, we are searching for some reasonably 
parsimonious set of environmental characteristics that have a 
bearing on what happens to students. In the process of doing this 
we are trying out some new measures of such characteristics as 
learning styles, peer-group patterns, discipline and freedom, and 
so forth; and we are deriving various indexes for such possibly 
useful variables as institutional diversity, teaching orientation, 
involvement in campus life, and cosmopolitanism of student 
body. 

There is a vast amount of published information, and data 
from test responses, that should be carefully examined. It is quite 
probable that the essence of this information can be reported in a 
relatively small number of crucial variables. But no such effort 
at data reduction has yet been attempted. The reason for some 
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optimism that it can be done comes from the fact that, despite 
differences in method and content, different methods do not pro- 
duce conflicting (negatively related) results; and even with some 
variations in content, certain variables continually emerge. 
Overall, there is a set of positive intercorrelations among differ- 
ent data sources that “should” be related, which range generally 
from the .30’s to the .6o’s. Admittedly these arc not “high” cor- 
relations, but they are about as large as are the typical correla- 
tions between SAT scores and college grades. 

Collecting Judgments front Potential Users and Contributors 

It would not be sufficient to base an information exchange sys- 
tem solely on the results of some data-reduction process applied 
to the thousands of pieces of potentially useful and available in- 
formation. The ultimate selection of what information is made 
available, and in what form, should also be guided by the ex- 
pressed desires and suggestions of the potential consumers (appli- 
cants, parents, and counselors) and the judgments of those who 
would have to provide some of the information (current college 
students and college officials). 

From high school students, parents, and counselors one needs 
to find out (or be assured that he knows) what they really want 
to know, what things they consider or would want to consider in 
making a choice among colleges. It will probably be discovered 
that some of their questions cannot be answered— questions like 
“Will the credits be accepted by another institution?” which can 
only be answered by the “other” institution. Or “What per- 
cent of the students smoke marijuana?” (Who knows? Is it 
legitimate public information?) Some questions that can be an- 
swered are probably of interest to a great many people; others 
may be of interest to very few people. A scries of interviews and 
group discussions can help to identify and give some perspective 
to these consumer concerns. Structured and unstructured ques- 
tionnaires, both for high school students and for parents, could 
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be distributed at regional test centers. In the structured question- 
naires one could present various types of information and ask 
for some ratings of their relative importance; in the unstructured 
case, one could simply ask what sort of information they seek or 
think would be most helpful to them. Some studies of this kind 
have been made; but new studies are needed because the relevant 
questions and concerns today may not be the same as those re- 
ported by researchers 5 or 10 years ago. 

Similarly, from college students and college officials (faculty 
and administration) one needs to learn what it is about the col- 
lege that they think prospective students should know— that is, if 
someone is interested in coming to this place, what are some of 
the things he ought to know about it. Again, interviews, group 
discussions, and answers to structured and unstructured question- 
naires would be important sources of information. 

All this is not to say that consumer surveys should determine 
what information is put into the system; but it is to say that the 
questions that researchers ask and that result in assorted scores 
and indexes are not necessarily the same as the questions of most 
relevance to consumers. The inquiries and insights of both are 
important, so that the system produces answers people want, and 
also answers they need regardless of their awareness of the need. 

Developing Guides for Decisions about the System 

At the beginning of this brief, I used the words “relevant, fea- 
sible, impartial . . . and valid.” Information about the college 
should make sense theoretically, educationally, and practically. 
By theoretical sense I mean that it cannot be a miscellaneous as- 
sortment of data that some people think might be useful, nor a 
presentation of vast amounts of unstructured data. There need to 
be guiding principles for selecting and grouping information. By 
educational sense I mean that the information should be relevant 
to institutional purposes and to the larger purposes of personal 
development and enrichment. By practical sense I mean that the 
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information must be usable and interpretable by the people for 
whom it is intended. The result of a research and development 
program should be a system that makes sense in all these ways. 

Moreover, information in the system must be defensible in 
the sense of fairness and impartiality and reliability. If opinions 
or collective judgments are part of the system they must be based 
on honestly selected cross sections of reporters. If characteristics 
of the student body are part of the system they must be based on 
reliable and sytematically applied measures. If various pieces of 
information arc combined to produce a score or index, then the 
psychometric properties of the score must be known. If typolo- 
gies or broad classifications are part of the system, then there 
must be statistical support for them. 

Operating the System in Successive Stages 

One need not await the results of extensive research before put- 
ting parts of a system into operation. 

The computer systems that enable applicants to identify col- 
leges having various characteristics (such as their location, size, 
programs, scholarships, and so on) can be adopted now. An in- 
teractive system, such as the one David V. Ticdeman has con- 
structed, might be the recommended model for expanded appli- 
cation. Presumably the content or software that can be put into 
the system initially would be limited to public information ob- 
tained from catalogs, directories, and other documents. In fact, 
initially, as John Hersey suggests in his brief to the Commission 
on Tests (page 51), the software may have to be in printed form 
until the computer hardware is available. In any case, the public 
information— of the sort suggested in briefs to the Commission 
on Tests by Coleman (page 19) and Hersey (page 51)— can be 
“interpreted” so that applicants will be helped to understand 
what it means. These interpretations can be based on present re- 
search knowledge, including at least some comparative informa- 
tion about the range of differences that exist among institutions 
of different types. 
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While this initial information is being prepared for the sys- 
tem, a second stage of research can be started. I conceive this 
stage as being the consumers’ and producers’ surveys that have 
been previously suggested. They would be intended to provide 
guidance for further amplification of both the student and col- 
lege input. 

A third stage of the research, which can begin whenever the 
first two stages begin, but which will take somewhat longer to 
complete, is that of examining the vast range of variables, items, 
and scores currently in existence, for the purpose of arriving at a 
more compact and manageable set of variables about student 
bodies and college environments to use in the final system. This 
kind of information, psychological and educational, will ulti- 
mately give a more meaningful base for mutual choices. 

One needs to recognize that it may not be possible, or if pos- 
sible may not be desirable, to envision a system that says to a stu- 
dent “These six colleges will be best for you” or to a college that 
“These students will be most compatible with your institutional 
environment.” Whether continuity and homogeneity or whether 
change and diversity are more productive of personal growth 
and development and of institutional adaptability is a question to 
which present research offers no solid answer. The sort of “ad- 
vice” one puts into a system— whether advice to applicants or to 
colleges— must be considered with more than a little humility, 
with a great deal of conservative caution, and with due respect 
for the possible results of research that has not yet been con- 
ducted. This does not mean that there is nothing useful one can 
say. Rather, it means that a combination of frankness and humil- 
ity is precisely what ought to distinguish the College Board’s 
program from the programs of commercial or less professional 
agencies. 

The fact that college applicants are required to submit a good 
deal of information about themselves— answers to questions on 
an application form, scores on standardized tests of ability and 
achievement, and a record of high school courses and grades— 
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has had major educational, personal, and societal consequences. 
It was partly an awareness of these consequences that led to the 
creation of the Commission on Tests. If, now, the College Board 
develops a system of information about colleges— parallel in in- 
tent and comparable in scope to what colleges require to know 
about applicants— -it is reasonably certain that the result will also 
have major educational, personal, and societal consequences. 

Recommendations 

It is recommended that the College Entrance Examination Board; 

1 . Support a research and development program to assemble, 
evaluate, and interpret information about colleges, taking into 
account the kinds of issues, sources, and technical matters de- 
scribed in this brief. 

2. Regard this program as contributing, in stages, to the nec- 
essary software underlying a computerized information system. 

3. Present information developed from the program in some 
printed form to be determined by the College Board, pending 
its incorporation into a computerized system, so that as much as 
possible will be available to applicants at the earliest feasible 
time. 



Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
A Program for Providing Information about Colleges 
to Applicants 



It is recommended that the College Entrance Examination Board; 

1. Support a research and development program to assemble, 
evaluate, and interpret information about colleges, taking into 
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account the kinds of issues, sources, and technical matters de- 
scribed in this brief. 

Endorsed by 20 members: Abelson, Blocker, Carroll, Coleman, 
Cross, Foster, Friedenberg, Gordon, Hechinger, Hcrsey, Holtz- 
man, Hoy, Marland, Pace, Pearson, Prentice, Schlossberg, Shir- 
ley, Thresher, Ticdcman. 

2. Regard this program as contributing, in stages, to the neces- 
sary software underlying a computerized information system. 

Endorsed by 20 members: Abelson, Blocker, Carroll, Coleman, 
Cross, Foster, Friedenberg, Gordon, Hechinger, Hcrsey, Holtz- 
man, Hoy, Marland, Pace, Pearson, Prentice, Schlossberg, 
Shirley, Thresher, Tiedcman. 

Foster: “If it becomes possible.” 

3. Present information developed from the program in some 
printed form to be determined by the College Board, pending its 
incorporation into a computerized system, so that as much as 
possible will be available to applicants at the earliest feasible time. 

Endorsed by 20 members: Abelson, Blocker, Carroll, Coleman, 
Cross, Foster, Friedenberg, Gordon, Hechinger, Hersey, Holtz- 
man, Hoy, Marland, Pace, Pearson, Prentice, Schlossberg, Shir- 
ley, Thresher, Ticdeman. 

General Comments 

Hersey: “I strongly support these recommendations. I am trou- 
bled, however, by the thrust of the passage in paragraph four 
(page 88). The fact that real choices arc at present available to 
only a few students seems in the quoted passage to be accepted 
as not only the way things are but also the way they have to be 
(‘Distinctive institutions and distinctive individuals will just have 
to find one another. . . It seems to me that the task before the 




98 



College Board is to provide every student choices of at least the 
magnitude of those every college now has. There is an important 
difference between telling students ‘where to go and what is 
good for them’ and telling each student where lie may be able 
to find what he wants and giving him as many clues as possible 
to what may be realistic for him to want. (This distinction ought 
to apply also to the last paragraph on page 96. What I’d hope the 
Board would say to the student is, ‘If you haven’t already, you’ll 
probably want to look into these six— or eight or ten— colleges, 
because they offer, in varying degrees and ways, which we’ll try 
to specify for you, things you appear to us to want or need.’ And 
so on.) As to the paragraph following the quoted passage, I 
doubt if there can be a generalized rank order of distinctions be- 
tween various categories of institutions; to the individual student 
one sort may be more important than another, and it would be 
precisely the task of an information system to point each student 
toward the distinctions that matter to him.” 

Shirley: “This is certainly the kind of information that needs 
to be researched before implementing a program of disseminat- 
ing information about colleges. We must be sure symmetry is 
possible and at least partially realistic. I would suggest, however, 
that the College Board might still be of assistance in getting sup- 
port for such an effort, without taking on the full expense of 
what is likely to be a more demanding program than its present 
resources or membership institutions would support. But foun- 
dation support should be sought to get these early steps done in 
preparation for implementation of a reasonable program to ob- 
tain symmetry.” 

Thresher: “I could hope for more detail on the problem of 
the large, complex institution.” 
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Education and Testing 

in a Period of Qualitative Change 

Richard Pearson 

The tests used in American education are viewed in different 
ways by different people. A teacher look on tests as an instruc- 
tional aid. A counselor expects tests to give him some insight 
into the promise that a young individual may have for some 
future endeavor. An admissions officer counts on tests to help 
him constitute an incoming class. A psychologist uses tests in 
experimental situations to validate hypotheses and to erect the- 
oretical structures. A company that sells tests looks on them as a 
product with market potential and financial value. And the stu- 
dent who takes tests sometimes sees them as suggesting oppor- 
tunities but increasingly often looks upon them as barriers that 
may or may not be overcome. 

Taking a broader perspective, tests can also be viewed as in- 
struments of educational and public policy. Their large-scale use 
at schools, colleges, and universities has important long-term 
effects on the size and composition of the student population, on 
student movement through the educational system, and on the 
substance of the educational process. Further, to the extent that 
education is also a public enterprise in the United States, the tests 
used in education arc also instruments of public policy that help 
determine what students will be educated in what programs for 
what future roles in American society. 

This paper is concerned with tests as instruments of educa- 
tional and public policy. More specifically it is concerned with 
the College Entrance Examination Board’s policy objectives in 
testing during the period of qualitative change that lies ahead 
during the 1970s. 

Today, higher education in the United States is facing great 
pressures for qualitative change. Student unrest on the campuses 
has directed the attention of faculty and administrators to the 
substance of education in the undergraduate programs. Militant 
black students are, on the one hand, asking colleges to admit 
young people who would not have been considered for college 
admission a decade ago. On the other hand, these same militant 
young blacks arc asking college faculties to offer courses and 
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programs in black studies for which much of the basic underly- 
ing scholarship has yet to be done. The Carnegie Commission on 
Higher Education is examining the functions, structure, organi- 
zation, and financing of the various institutions of higher educa- 
tion. In considering these various sources of pressure for qualita- 
tive change, Lewis B. Mayhew concludes, “It becomes neces- 
sary, then, only to anticipate whether the changes will have been 
evolutionary and deriving from present practice, or revolution- 
ary to an extent that the entire character of undergraduate edu- 
cation will have been altered.” 1 

Some sense of the qualitative nature of the impending changes 
is given by Abraham H. Maslow writing about the development 
of a ‘Third Force” psychology within that discipline. Maslow 
writes as follows: ‘The upshot of the past decade or two of tur- 
moil and change within the field of psychology can be viewed 
as a local manifestation of a great change taking place in all fields 
of knowledge. We are witnessing a great revolution in thought, 
in the Zeitgeist itself: the creation of a new image of man and so- 
ciety and of religion and science. It is the kind of change that 
happens, as Whitehead said, once or twice in a century. This is 
not an improvement of something; it is a real change in direction 
altogether. It is as if we had been going north and arc now going 
south instead.” 2 

Qualitative change will, of course, be accompanied by quan- 
titative change in higher education as enrollment predictions 
continue to mount. Sidney G. Tickton projects an increase of 
103 percent in college enrollments between 1965 and 1980. 3 In 



1. Mayhew, Lewis B., "The Future Undergraduate Curriculum,” in Alvin C. 
Eurich, cd., Campus 1980 , the Shape of the Future in American Higher Education , 
New York: Dclacortc Press, 1968, p. 202. 

2. Maslow, Abraham H., “Some Educational Implications of the Humanistic 
Psychologies.” Harvard Educational Review, Fall 1968, p. 685. 

3. Tickton, Sidney G., “The Magnitude of American Higher Education in 
1980,” in Alvin C. Eurich, cd., Campus 1980 , The Shape of the Future in Ameri- 
can Higher Education. New York: Delacorte Press, 1968, Table C, p. 14. 
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response to predictions of this kind, the Carnegie Commission 
on Higher Education calls for the establishment of 500 new com- 
munity colleges and 50 urban four-year colleges by 1976, to meet 
the needs of the inner cities. 4 If experimentation and qualitative 
change are in the air, then these new institutions may well be 
the places where dramatic changes will occur. 

Perhaps the best balanced summary of needed change in 
higher education is given by John W. Gardner. Gardner’s agenda 
has eight important items, the first two of which deal with teach- 
ing and with the undergraduate curriculum. In the latter con- 
nection he writes as follows: 

“The colleges and universities arc going to have to under- 
take a thoroughgoing reform of the undergraduate curriculum. 
We have now had a decade of lively reform in the high-school 
curriculum. A comparable movement for reform at the college 
level is already under Way, and we shall be hearing a great deal 
more about it in the years ahead. It is certain to transform in- 
struction in all major fields of knowledge. It will require search- 
ing reappraisal of the aims of education in each field. It will re- 
quire thorough exploration of the possibilities of new teaching 
aids and methods. It will involve a more widespread and in- 
genious use of independent study. And it must involve a con- 
tinuing effort to do justice to interdisciplinary approaches. Cur- 
riculum reform will be incomplete if its only consequence is that 
each specific subject is better taught; it must also reintroduce into 
the undergraduate program the breadth so essential for young 
people who will reach the peak of their careers in the twenty- 
first century.” 5 

These various expectations for qualitative change in higher 

4. Carnegie Commission on Higher Education, Quality and Equality: New Lev- 
els of Federal Responsibility for Higher Education. New York: McGraw-Hill, Dec. 
1968, p. 37. 

5. Gardner, John W., “Agenda tor the Colleges and Universities,” in Alvin C. 
Eurich, cd., Campus 1980, The Shape of the Future in American Higher Education. 
New York: Delacorte Press, 1968, p. 2. 




102 



education present the College Board with at least three impor- 
tant implications for its testing function. First, college programs 
can be expected to call for increasingly sophisticated abilities on 
the part of students. A decade of strong preparation for college 
in the country’s outstanding secondary schools has advanced the 
level of performance of beginning college students on the com- 
petitive campuses to a level one to two years beyond the students 
of an earlier day. Independent study and major research papers 
arc now an aspect of the secondary school curriculum and result 
in the development of skills that were formerly developed only 
in college. This calls for greater sophistication and greater flexi- 
bility in the Board’s tests that will be used in conjunction with 
competitive admission. Second, educational programs at the col- 
lege level can be expected to increase greatly in variety and di- 
versity. The current emphasis on urban colleges and universities 
will undoubtedly give great impetus to the organization of new 
educational programs in the health sciences, social services, urban 
education, and other fields that arc related to improving the 
quality of life in urban areas. Traditional academic subjects in 
secondary school will be replaced at least in part by preparatory 
courses for these newer fields. Third, as has been made clear on 
many campuses and in many high schools, students will expect 
to participate more directly in the decisions that affect them and 
their future. In order to participate, they will need sound, scien- 
tifically based information while dicy are still in secondary 
school. 

The fact of strong student interest in involvement and par- 
ticipation is one that the Board should respond to with all de- 
liberate speed. The Board’s testing function should be focused 
principally on the high school student during the middle school 
years and on the plans that he or she needs to make in connection 
with continued education beyond high school. The wisdom of 
this step is further supported by signs of increased maturity 
among young people today, the increasing specialization in 
many college programs, and the limited guidance resources 
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available in many schools. The evaluation of past educational 
progress and the estimation of future educational performance 
thus can be, as it should be, the student’s own business far more 
than has been the case previously. 

Accordingly my first recommendation is as foliows: 

i. The Board should adopt as its principal policy objective in 
testing the provision of a broad array of tests and other informa- 
tion designed to assist tenth- and eleventh-grade high school 
students in educational planning and decisions with respect to 
post-high school opportunities. 

Acceptance of this policy objective implies a considerable ex- 
tension of tests and other information beyond what is now avail- 
able in the Board’s programs. The high school population of the 
country is made up of young people whose abilities and achieve- 
ments arc more varied than the Board has traditionally recog- 
nized. Further, the existence of some young people in this popu- 
lation whose abilities have not been adequately developed in the 
schools requires that special provision be made for them. The 
suggestion by Edmund Gordon and Winton Manning for the 
redirection of testing efforts for disadvantaged children is indica- 
tive of the breadth that is required. They have proposed the de- 
velopment of tests for this population that would serve prescrip- 
tive functions— that is, tests that would help prescribe the kinds 
of instructional programs that would be expected to overcome 
particular educational deficits . 6 The need for tests predictive of 
performance in the vocational, technical, and subprofcssional 
programs of the community colleges is another example of the 
extension required. 

In connection with this first recommendation, it is important 
to emphasize that I am proposing broader criteria for the inclu- 
sion or exclusion of individual tests in the Board’s overall pro- 
gram than have previously been utilized. The underlying cri- 



6. Manning, Winton H., ‘‘The Measurement of Intellectual Capacity and Per- 
formance.” Manuscript submitted to The Journal of Negro Education. 
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terion implied by this recommendation is the student’s own 
interest in planning his educational career. The Board’s present 
tests arc focused largely on institutional criteria. Thus low test 
performance on the Scholastic Aptitude Test (sat) constitutes a 
caution to the college in extending an admissions offer but is of 
little use to the student who needs information about the educa- 
tional program he might next consider in order to develop fur- 
ther the abilities and interests he may have. Acceptance of a stu- 
dent-oriented criterion for the Board’s tests also implies extend- 
ing validation of the tests beyond the criterion of first-year 
grades in college. The use of grades as criteria for college en- 
trance tests has traditionally been in the interests of the institu- 
tions. And, while there is some convergence between institu- 
tional interest and student interest, the student’s interests in 
educational planning would be better served if tests were also 
validated against success in the careers made possible by the 
educational program in question. A third implication of this ex- 
tension of the criterion for the Board’s tests is that it is in the 
interests of some high school students to make comparisons be- 
tween opportunities for further education and opportunities for 
employment immediately upon graduation from high school. 
Such comparisons arc not greatly facilitated at the present time 
by the Board’s exclusive interest in college admissions. 

With further respect to student involvement, the amount of 
information available to colleges and universities about students 
is greater than the amount of information about the institutions 
that is available to prospective applicants. The scarcity and in- 
accuracy of information about college programs and the campus 
environment is a serious handicap to full student involvement in 
decisions about college attendance. Accordingly my second rec- 
ommendation is as follows: 

2. The Board should provide factual and scientifically based 
information about individual colleges and universities for use by 
prospective applicants. This information should go beyond that 
now routinely supplied by institutions of higher education and 
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should include ability and performance expectations for admis- 
sion and for success in particular programs as well as information 
about the campus environment. 

This recommendation calls for the provision of information 
by colleges that they are not now providing either directly or 
through such sources as The College Handbook and the Manual of 
Freshman Class Profiles. The justification for such a recommenda- 
tion in this context is one of equity in the institution’s dealings 
with the individual student in the admissions process. Justifica- 
tion lies more generally, however, in the area of public account- 
ability and the responsibility of institutions of higher education 
to report generally in the public interest. 

The institutional functions of admissions and placement at 
the colleges and universities require utilization of some, although 
perhaps not all, of the information that would be made available 
to high school students under recommendation i above. Thus 
the Board should make provision for the reporting of test results 
to the appropriate offices of the colleges and universities, on re- 
quest of the individual applicants. I visualize this process operat- 
ing under a scries of three to five specialized reporting programs 
where the data required would coincide with the admissions re- 
quirements of the particular college or university. Thus a com- 
prehensive university with a number of schools and colleges and 
a variety of educational programs might L. /c different require- 
ments from a liberal arts college. Similarly a four-year institution 
might have different requirements from a community college. 
Eventually, when computer use is more widespread and when 
the test information called for under Recommendation i can be 
stored in legional or local data banks, this information can be 
made available to the higher education institutions “on call” 
from the computer. Accordingly my third recommendation is 
as follows: 

3. The Board should make provision for a series of three to 
five specialized reporting programs, corresponding to different 
types of institutions of higher education, for the provision of test 
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results to colleges and universities in connection with institu- 
tional decisions about admissions and placement. Further, experi- 
mentation should be undertaken with respect to computer stor- 
age and retrieval of this information so that it can be made 
available to the institutions at the time and in the form needed 
for decisions. Further, standards for protecting the privacy of 
students should be established so that any test information neces- 
sary for students but not appropriate for use in institutional de- 
cisions will be reported only to students. 

The provision of a broad array of tests and other information 
for educational planning and guidance in the high schools and 
the further provision of reporting programs that would place 
relevant parts of the test results in the hands of the colleges and 
universities for institutional decisions should reduce redundancy 
in testing that exists today and should provide adequate infor- 
mation for both students and institutions. There may be in- 
stances, however, where this global approach may leave unmet 
certain institutional requirements and the requirements of certain 
groups of students. The relatively small number of highly com- 
petitive colleges and universities may, for example, require spe- 
cialized testing for admissions decisions beyond that provided for 
under Recommendation i above. And because this recommen- 
dation is focused on students while they are in high school, there 
may be individuals who are prospective applicants for college 
admission who are not part of the high school population. For 
both of these reasons it may be necessary to provide specialized 
testing programs for limited numbers of institutions or for lim- 
ited numbers of individuals. These specialized programs should 
be introduced only when there is clear justification for them and 
only when the basic program is clearly inadequate in some re- 
spect. Accordingly my fourth recommendation is as follows: 

4. The Board should be prepared to offer supplementary 
testing programs for particular groups of colleges and univer- 
sities and for prospective applicants who may not be enrolled in 
high school under provisions that would limit redundancy and 
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duplication with the basic program. 

All the preceding recommendations should move the Board 
away from the current strong emphasis on selective admissions 
by institutions to a strong emphasis on educational planning and 
decision by students who make up a comprehensive and diverse 
high school population. Further, the breadth of test offerings in 
the basic program proposed in Recommendation i should sup- 
port multiple channels of access to the variety of institutions that 
will characterize the colleges and universities of the future. It is 
likely, however, that the pace and timing of qualitative change 
among higher education institutions will be uncertain and that 
this uncertainty will pose continuing serious policy problems for 
the Board. Consequently my fifth recommendation is as follows: 

5. The Board should keep the objectives proposed in this re- 
port under continuing policy review and provide adequate em- 
pirical research in connection with this continuing review in 
order to assess the extent and timing of the proposed changes. 

In conclusion, the overall thrust of the foregoing recommen- 
dations is to place the Board’s present testing activities in a flex- 
ible position, one that will make them responsive to impending 
qualitative change within higher education. The direction of 
this change is toward an inclusive system of colleges and uni- 
versities with multiple channels of access— a system that will no 
longer exclude substantial proportions of American youth. Tests 
of the future must be at least as efficient in selecting young peo- 
ple into higher education as traditional tests have been in select- 
ing them out. 

Recommendations 

It is recommended that the College Entrance Examination Board: 

1. Adopt as its principal policy objective in testing the provi- 
sion of a broad array of tests and other information designed to 
assist tenth- and eleventh-grade high school students in educa- 



tional planning and decisions with respect to post-high school 
opportunities. 

2. Provide factual and scientifically based information about 
individual colleges and universities for use by prospective ap- 
plicants. This information should go beyond that now routinely 
supplied by institutions of higher education and should include 
ability and performance expectations for admission and for suc- 
cess in particular programs as well as information about the 
campus environment. 

3. Make provision for a series of three to five specialized re- 
porting programs, corresponding to different types of institu- 
tions of higher education, for the provision of test results to col- 
leges and universities in connection with institutional decisions 
about admissions and placement. Further, experimentation 
should be undertaken with respect to computer storage and re- 
trieval of this information so that it can be made available to the 
institutions at the time and in the form needed for decisions. 
Further, standards for protecting the privacy of students should 
be established so that any test information necessary for students 
but not appropriate for use in institutional decisions will be re- 
ported only to str dents. 

4. Be prepared to offer supplementary testing programs for 
particular groups of colleges and universities and for prospective 
applicants who may not be enrolled in high school under pro- 
visions that would limit redundancy and duplication with the 
basic program. 

5. Keep the objectives proposed in this report under continu- 
ing policy review and provide adequate empirical research in 
continuing review in order to assess the extent and timing of the 
proposed changes. 
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Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
Education and Testing in a Period of Qualitative Change 

It is recommended that the College Entrance Examination Board: 

1. Adopt as its principal policy objective in testing the provision 
of a broad array of tests and other information designed to assist 
tenth- and eleventh-grade high school students in educational 
planning and decisions with respect to post-high school oppor- 
tunities. 

Endorsed by 18 members : Abelson, Blocker, Carroll, Coleman, Fos- 
ter, Friedenberg, Gordon, Hechinger, Hersey, Holtzman, Hoy, 
Marland, Pace, Pearson, Schlossberg, Shirley, Thresher, Tiede- 
man. 

Foster: “I think it is important to change the reference to as- 
sisting ‘tenth- and eleventh-grade high school students’ to ‘col- 
leges and prospective students.’” 

Tiedeman: ‘‘I would prefer more ambiguous words than 
‘tenth- and eleventh-grade.’” 

Not endorsed by 1 member : Prentice. 

2. Provide factual and scientifically based information about in- 
dividual colleges and universities for use by prospective appli- 
cants. This information should go beyond that now routinely 
supplied by institutions of higher education and should include 
ability and performance expectations for admission and for suc- 
cess in particular programs as well as information about the 
campus environment. 

Endorsed by 18 members: Abelson, Blocker, Carroll, Colernan, Fos- 
ter, Friedenberg, Gordon, Hechinger, Hersey, Holtzman, Hoy, 
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Marland, Pace, Pearson, Schlossberg, Shirley, Thresher, Tiede- 
man. 

Not endorsed by i member: Prentice. 

3. Make provision for a series of three to five specialized report- 
ing programs, corresponding to different types of institutions of 
higher education, for the provision of test results to colleges and 
universities in connection with institutional decisions about ad- 
missions and placement. Further, experimentation should be 
undertaken with respect to computer storage and retrieval of 
this information so that it can be made available to the institu- 
tions at the time and in the form needed for decisions. Further, 
standards for protecting the privacy of students should be estab- 
lished so that any test information necessary for students but not 
appropriate for use in institutional decisions will be reported 
only to students. 

Endorsed by 18 members: Abelson, Blocker, Carroll, Coleman, Fos- 
ter, Friedenberg, Gordon, Hechinger, Hersey, Holtzman, Floy, 
Marland, Pace, Pearson, Schlossberg, Shirley, Thresher, Tiede- 
man. 

Not endorsed by 1 member: Prentice. 

4. Be prepared to offer supplementary testing programs for par- 
ticular groups of colleges and universities and for prospective 
applicants who may not be enrolled in high school under pro- 
visions that would limit redundancy and duplication with the 
basic program. 

Endorsed by 18 members : Abelson, Blocker, Carroll, Coleman, Fos- 
ter, Friedenberg, Gordon, Hechinger, Hersey, Holtzman, Hoy, 
Marland, Pace, Pearson, Schlossberg, Shirley, Thresher, Tiedc- 
man. 
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Thresher: “Recommendation 4 may require some vigilance 
to prevent the new resurgence of elitism in groups of colleges 
seeking special tests. This could nullify or tend to nullify Car- 
roll’s proposal to reduce redundancy in testing.” 

Not endorsed by 1 member: Prentice. 

5. Keep the objectives proposed in tliis report under continuing 
policy review and provide adequate empirical research in con- 
tinuing review in order to assess the extent and timing of the 
proposed changes. 

Endorsed by i8 members: Abclson, Blocker, Carroll, Coleman, Fos- 
ter, Friedenberg, Gordon, Hechinger, Hersey, Holtzman, Hoy, 
Marland, Pace, Pearson, Schlossberg, Shirley, Thresher, Tiedo 
man. 

Not endorsed by t ntetnlfer: Prentice. 

General Comments 

Foster: “In speaking of ‘abilities and interests fa student] may 
have,’ we should be mindful in t* Mscussion about relevance 
and student choice (and both are essential) of the once-vaunted 
purpose of the American college to help a student discover in- 
terests and abilities he knows not of 

“In the paragraph preceding his fifth recommendation Pear- 
son mentions moving the Board away from the current strong 
emphasis on selective admissions. Change in the universities is 
essential and will take place. Not every one of the 100 percent of 
youth who should have ‘higher’ education will profit most by 
strict and strictly intellectual education, but many students and 
many aspects of society can. Therefore, endorsement of needed 
changes should not, even by omission, imply lower valuation of 
intellectual forms of endeavor or the desirability for some selec- 
tivity of those who can most profit from and enjoy such en- 
deavor.” 
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Hersey: “I support these recommendations on the assumption 
that they arc not exclusive— for example, endorsing the third 
recommendation, that experimentation be undertaken with 
computer storage of student test results, should not exclude the 
possibility of computer storage of data on colleges for distribu- 
tion to students.” 

Shirley: “I am totally sympathetic with these objectives, but 
I fear the cost of their implementation has not been studied. Who 
is to pay? Knowing something of the magnitude of such national 
programs, I am convinced that neither the students (and their 
parents) benefited nor the high school and college members of 
the College Board could possibly support such an ambitious 
program. I should request that the Board (a) make a proposal to 
a national foundation to explore in depth the costs of such a pro- 
gram and (b) make strong recommendations to government (the 
Department of Health, Education, and V/elfare) to implement it 
on a pilot level and, as experience grows, on a national basis.” 
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Regional Centers for Guidance 
in Continuing Education 

Nancy K. Schtossberg.John C. Hoy , and Edmund W. Gordon 



Higher education, in providing expanded opportunities, is 
heavily burdened by having to develop services for the enor- 
mous variety of people who need and want to be served. 
Whether the needs of individuals seeking post-high school op- 
portunities or the institutions offering such opportunities are 
being analyzed, a persistent issue is the process by which an indi- 
vidual makes a sound and effective decision concerning his own 
educational development. This process affects all people, whether 
they arc over privileged, disadvantaged, of minority status, 
adults, students who want to work and study part time, or tradi- 
tional students interested in a two- or four-year college program. 
The present pattern of procedures rends to favor and reward a 
student who has developed within the confines of the system and 
who has made no (or only a few) mistakes. Indeed, the students 
who find the present admissions system working to their advan- 
tage are those who have attended and seek to attend the elite 
schools and colleges. But even among this group criticism of the 
system is vociferous. 

As the number and variety of interested people mount, there 
arc more and more problems related to providing guidance serv- 
ices to those in the process of making sound decisions about their 
educational development. These problems are greatly exacer- 
bated when guidance and decision making involve people and 
adults who are exceptional or who are consideied “offbeat.” 
The problem of individualized guidance or alternatives to 
guidance will undoubtedly increase if the present trend toward 
general disaffection with the education establishment continues 
to grow. Institutions of higher education, burdened as they are, 
are not likely to change rapidly enough to accommodate the 
thousands of individuals who, although they deplore the present 
system, still wish to use it— especially those who view their own 
unique experience and background as an asset rather than a hin- 
drance to educational acceptability and growth. 

In a very profound sense, the College Board has so far been 
buffered from the major assault oil traditional procedure and 
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structure that its institutional membership has experienced over 
the last two years. This attack will reach the College Board in 
time, but it is important for the Commission on Tests to suggest 
deliberate steps that can be taken now to make the College 
Board both more relevant to the changing needs of people and 
to the way they view access to higher education, as well as to 
facilitate programs that deal with the atypical, the eccentric, the 
disadvantaged, the neglected, and the growing number of 
people who simply choose to take an idiosyncratic route to their 
educational goal. 

Though the College Board’s role is well defined and seem- 
ingly necessary to present patterns of access to higher education, 
the question the Commission on Tests should resolve is whether 
the organization’s resources— both financial and human— can be 
used to increase the possibility of permitting and encouraging all 
.tudents to have more personal initiative in the process of pur- 
suing a wider pattern of educ ational goals. 

During the next decade the nation will sec the development 
of continuing- and adult-eaucation programs that culminate in 
degrees, certificates of advanced study, and licensing in a variety 
of vocational fields. Though the College Board has in a number 
of peripheral ways been responsive to the evolution of open 
door programs, it has not assumed clear leadership in the ex- 
tension of its services to people and institutions directly related 
to this significant shift in the structure of higher education and 
its affiliates. Rather, the College Board has tended to make more 
efficient and comprehensible the present pattern of access to 
higher education through emphasis on selective admissions pro- 
cedures for selective institutions or those that would like to be 
thought selective. As such, the Board has operated as a part of 
the system, an arm of the establishment. As our educational sys- 
tems become more complex and inclusive, there is a growing 
need for an advocativc, protective, and supportive agency out- 
side the established institutional structures. This agency world 
be directed at insuring that the best interests of the client arc at 
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all times protected, just as agents of institutional administration 
have as a prime responsibility the protection of the institution’s 
best interests in behalf of all clients. It would be based on the 
legal conception of the ombudsman— one who helps and even 
protects an individual citizen in his encounters with agents of 
government. The guidance agency would be the client’s advo- 
cate and protector— insuring, at every point in the process of 
education decision making and implementation, that his best 
interests are served. 

A major emphasis must be placed on developing new pat- 
terns of service which can be built on but ought not 10 be merely 
extensions of the present College Board programs and services. 
The following suggestions are primarily illustrative but should 
serve to make clear the emphasis sought. 

I. The College Board should establish regional centers for 
guidance in continuing education. These centers should be es- 
tablished through cooperative relations and support between 
public and private schools, technical institutes, colleges, and 
community groups. However, the centers should be independ- 
ent of the control of the education institutions. The initial focus 
would be on urban locations such as New York, Washington, 
Chicago, and Los Angeles. 

Board leadership should be provided; however, the centers 
should place priority on grassroots support and should encourage 
local variations on the College Board program. Funding for such 
centers would be gathered from several sources: the College 
Board colleges and universities, school systems, the United 
States Office of Education, foundations, student or candidate 
fees, local taxes. 

The centers would provide opportunities for field research 
and would offer, through cooperative arrangements with uni- 
versity member institutions, graduate internships in guidance 
and counseling. The centers would also emphasize in-service 
training of counselors at the secondary school and college level. 
In addition, non-cducation-based counselors or guidance special- 
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ists would be trained to handle primarily clients with placement 
rather than counseling problems. 

Regional centers would receive support on the level now 
offered to College Board regional office programs. Direction of 
the new program should be fully integrated with present Board 
regional office activities, though they should not be subordinated 
to the new program. 

2. The College Board should consider as potential programs 
for these centers: 

A. Offering appraisal and advisement services. Some of the 
wide variety of procedures that should be available are listed 
below. Formal testing should be used with discretion. All assess- 
ment should be directed at the development of a qualitative de- 
scription of the client’s function, potential, and needs. 

(1) Individual counseling. The opportunity for individuals 
to talk with trained counselors would be available. Working 
schedules would be arranged so that professionals and parapro- 
fessionals would be free to consult with people who simply walk 
in ready to talk to someone. The other staff could schedule ap- 
pointments for people who want to return. 

(2) Group guidance. Opportunities would be provided for 
people to meet in groups to discuss what they want to do with 
their lives and how to implement their plans. The sessions would 
be scheduled regularly, and a client could choose to join a group, 
sec a counselor individually, or both. 

Special groups would also be scheduled and advertised. One 
structured group could revolve around simulation career games 
such as the Life Career Games developed by James S. Coleman 
and Sarancs Boocock of the Johns Hopkins University. These 
games require a trained leader, and the players must meet to- 
gether several times. The games enable the player to simulate 
life and career planning by making game decisions, seeing their 
consequences, and recognizing the need to plan in many areas as 
a basis for self-development. Other groups may vary in pattern 
from unstructured brainstorming or bull sessions, through a va- 
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riety of approaches, to guided group interaction, including 
formal didactic orientation sessions. 

(3) Phone counseling. Many people, but especially adults, 
have concerns but cannot bring themselves to come to a coun- 
selor or agency. A trrined person would be available at all times 
to talk at length on the phone. As a result of these conversations, 
adults might later come to the center or meet with one of the 
mobile teams. 

(4) Browsing room for guidance resources. Bulletin boards, 
files, books, and films about training and educational and occu- 
pational opportunities would be accessible and attractively lo- 
cated. The room would be an attractive and hospitable place for 
members of the community to browse and think about them- 
selves. Coming to the browsing room would not automatically 
mean that counseling or testing or both would result. Many 
people need guidance resources but not guidance counselors. 

(5) Computer-assisted guidance facilities. The Information 
System for Vocational Decisions by David V. Ticdeman, the 
Educational & Career Exploration System currently under de- 
velopment by Donald E. Super, or some other such system 
might well be accessible through terminals placed in these 
centers. 

(6) Mobile counseling teams. Mobile teams of professional 
and paraprofcssional guidance workers and counselors would 
systematically visit neighborhood centers, housing projects, 
schools, and other social agencies and training centers. These 
mobile units would provide the reaching out that so many 
youths and adults need, especially those from the disadvantaged 
areas. 

B. Developing a system of credit equivalency. The growing 
number of adult students reentering college makes it necessary 
to develop a credit equivalency program. For example, some 
faculty members of Wayne’s College of Education have been 
meeting with officials of several poverty programs and commu- 
nity college representatives to work out a system of credit cquiv- 
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alency for paraprofcssionals who engage in noncredit training 
and then decide to pursue academic programs. The mechanics 
of this program arc being worked out, and models from several 
institutions will soon be available. 

C. Offering courses in guidance for credit. A series of one- 
credit guidance courses would be developed and offered through 
the departments of educational guidance and counseling and 
urban affairs of the institutions concerned. These courses would 
be unique: First, by the offer of credit and a systematic way to 
go about locating resources, youths and adults would be stimu- 
lated to come to the center. Second, the course would have two 
instructors: a member of the center’s professional staff and a 
graduate student who had been in either Upward Bound or the 
Higher Education Opportunities Program. Minimal fees would 
be charged. The actual nature of the courses would, of course, 
depend on the nature of the clients. Some proposed courses 
would be: How to apply to college, the nature of higher educa- 
tion (types of colleges, admissions procedures); How to locate 
vocational resources (scholarships available, good technical insti- 
tutes training beauticians, barbers, skilled workers, and so forth); 
How to study (methods of study, blocks to studying). 

D. Using new training methods for staff development. Ad- 
vanced doctoral students in educational counseling would run 
the center. This would serve as the practical experience required 
of most doctoral students in guidance and counseling depart- 
ments. They would provide the center with “professionals in 
training” at all times, as well as provide high quality counselors 
at a minimal cost. Paraprofcssionals (not people studying to be 
counselors, but neighborhood leaders) would be recruited and 
trained. One paraprofessioual would serve on each mobile team 
as the “contact” or “intake” person. Paraprofcssionals would 
staff the browsing room and handle phone interviewing. 

E. Conducting experimentation and research. If better op- 
portunities arc to be made available to people not well served by 
higher and continuing education, more progress must be made 
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in dealing with late adolescents and adults who have educational 
deficiencies or learning problems. Colleges and post-high school 
training institutions have moved too slowly toward a solution, 
and there is a critical need for some agency to assume respon- 
sibility for research in this area The regional centers might 
well institute experimental programs intended to assure success 
in continuing education for a randomly selected group of high- 
risk candidates. This aspect of the program should be concerned 
not only with remedial education, but also with correction of 
learning disabilities, circumvention of educational handicaps, 
attitudinal and motivational rehabilitation, and modified condi- 
tions of learning, as well as changed outcomes for learning. The 
program would be experimental because, even though several 
existing programs— such as the Job Corps, Mobilization for 
Youth, Upward Bound, and collegiate programs for disadvan- 
taged youngsters— have dealt with aspects of the problem, none 
have been all-encompassing or highly successful. The College 
Board centers should take on the more basic problem of the 
feasibility of changing achievement and productivity patterns of 
young adults whom society has cither inadequately served or 
damaged. 

3. The College Board should incorporate student participa- 
tion into the regional centers and into the Board's national pro- 
gram. As beneficiaries of the program, students would obviously 
participate in it. In addition, secondary school and college stu- 
dents could be employed at a regional center part-time, could 
serve in internship roles during the summer, and might receive 
undergraduate and graduate credit at the institutions that co- 
operate in the respective regional groups. Students should also 
be active in the governance of these centers. Provision must be 
made for them to be represented on policy boards and, if pos- 
sible, to participate in the administration of programs. 

4. The College Board should change its name and function 
to “Continuing Education Entrance Board.” This implies that 
the focus would be to assist everyone in getting into the educa- 
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tional vocational opportunity structure. 

Over the next decade significant steps should be taken to 
broaden the Board’s constituency to include the wide variety of 
institutions concerned with post-high school development as 
well as a constituency of self-directed continuing-education stu- 
dents. 

It seems abundantly apparent that the Commission on Tests 
is substantially behind the notion of developing symmetry in the 
college selection process. Field centers will be a necessary part of 
any such development. These recommendations are made so 
that such activity may begin without delay. By utilizing present 
and contemplated Board programs as the base for more direct 
contact with a wide cross section of candidates, counselors, and 
the general public, an imaginative and pace-setting operation 
should be possible. In turn, such an enterprise should prove to be 
an excellent vehicle for future developments in client-oriented 
programs related to ocher recommendations of the Commission. 

Recommendations 

It is recommended that the College Entrance Examination Board: 

1. Establish regional centers for guidance in continuing edu- 
cation. 

2. Consider as potential programs for these centers: 

a. Offering appraisal and advisement services, 

b. Developing a system of credit equivalency, 

c. Offering courses in guidance for credit, 

d. Using new training methods for staff development, 

c. Conducting experimentation and research. 

3. Incorporate student participation into the regional centers 
and into the Board’s national program. 

4. Change its name and function to “Continuing Education 
Entrance Board.” 
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Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
Regional Centers 

for Guidance in Continuing Education 



It is recommended that the College Entrance Examination Board: 

1. Establish regional centers for guidance in continuing education. 

Endorsed by 17 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedcnberg, Gordon, Holtznian, Hoy, Marland, Pace, 
Pearson, Prentice, Schlossberg, Shirley, Thresher, Ticdcman. 
Pace: “For one or two pilot demonstrations.” 

Not endorsed by 1 member: Hersey. 

2. Consider as potential programs for these centers: (a) offering 
appraisal and advisement services, (b) developing a system of 
credit equivalency, (c) offering courses in guidance for credit, 
(d) using new training methods for staff development, and, (e) 
conducting experimentation and research. 

Endorsed by 16 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedcnberg, Gordon, Holtznian, Hoy, Marland, Pace, 
Pearson, Schlossberg, Shirley, Thresher, Tiedeman. 

Pace: “Most of these services require a fairly complex and 
imaginative organization, and their success may depend as much 
on community, political, and economic factors and so forth as 
on educational ones.” 

Not endorsed by 2 members: Hersey, Prentice. 

Prentice: “Of the potential programs, I endorse only experi- 
mentation and research.” 

3. Incorporate student participation into the regional centers 
and into the Board’s national program. 



Endorsed by 15 members: Abclson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Holtzman, Hoy, Marland, Pace, 
Pearson, Schlossbcrg, Thresher, Ticdeman. 

Not endorsed by 3 members: Hersey, Prentice, Shirley. 

4. Change its name and function to “Continuing Education En- 
trance Board.” 

Endorsed by 9 members: Abclson, Blocker, Foster, Gordon, Hoy, 
Marland, Schlossbcrg, Thresher, Ticdeman. 

Thresher: “There might be troublesome problems if the pres- 
ent name were abandoned and left in the public domain. Per- 
haps it should be retained as a subsidiary.” 

Not endorsed by 8 members: Carroll, Coleman, Friedenberg, Her- 
scy, Holtzman, Pearson, Prentice, Shirley. 

Holtzman: “I would rather not seek a name change until 
some of the above new activities have been tested in the field, 
validated as to worth, and expanded operationally. Eventually 
such a name would be a good idea.” 

Pearson: “I agree with the desirability of a name change, but 
P111 not sure this is it.” 

1 member abstained: Pace. 

General Comments 

Abelson: “This brief, as well as the brief by Hersey (‘A College 
Entrance Service’) appears to 111c to include laudable objectives. 
However, when looked at in total, implementation of all these 
proposals by the College Board docs not seem feasible. The costs 
would be too great. If these recommendations arc to amount to 
more than rhetoric, further steps would be necessary. Either 
there must be a proposal for a large-scale federal subsidy or the 
Board should arrange a scale of priorities. In view of emerging 
Patterns of government interference with education, I would be 
against proposing federal subsidy.” 
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Carroll: “As with some of the other briefs, I endorse these 
recommendations only to indicate that some of them represent 
worthy goals, but I urge caution in view of the problems of 
financing and staffing.” 

Hersey: “As with Mar land's brief, I endorse without reserva- 
tion the authors' theses and their statement of the urgent need for 
the services described, but I feel that the College Board v/ill have 
its hands full to effect the symmetry we have discussed in its role 
of facilitating entrance to higher education. It would be valu- 
able, of course, to have the Commission endorse and support the 
proposal and recommend that others take up the task.'' 

Holtzman: “In this proposal testing is slighted to a greater 
degree than it needs to be." 

Marland: “Since this proposition calls for a very large change 
in procedures and a very complex structure, it is suggested that a 
prototype or two be developed for trial purposes before attempt- 
ing a full-blown program." 

Prentice : “I am troubled to find that the authors (all broadly 
experienced in educational guidance and assessment) can recom- 
mend a program that strikes me as totally unrealistic. I would 
estimate that the cost of the program proposed would be on the 
order of loo :o 1,000 times the present budget of the College 
Board, and the benefits would be only speculatively related to 
costs. I seriously doubt that enough ‘advanced doctoral candi- 
dates in educational rounseling' exist to supply more than token 
regional centers— or token service in ones that might be estab- 
lished. I urge that a more limited and more practical proposal be 
substituted for this one, with more thought being given to cost- 
benefit ratios and realistic sources of funds." 

Shirley: “Recommendation 2: There should be no firm or 
major commitment until (1) success of efforts, and (2) potential 
funding, are assured. There arc signs that the mood is rejection 
of, rather than acceptance into, the society by way of education." 
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Diversification in Educational Assessment 

B. Alden Thresher 



It is unrealistic to propose that the College Entrance Examination 
Board should simply cease testing for college admission, or even 
abruptly change its present procedures. The Board should rather 
start by taking seriously the shortcomings of the existing mode 
of testing, make a major project of developing alternative ap- 
proaches, and be prepared gradually to reduce the almost exclu- 
sive reliance on present testing modes and conventions. Gradual- 
ness will be enforced by the conservatism of the Board member 
colleges, even though the Board docs its best to speed up the 
pace. Above all, both the College Board and Educational Testing 
Service should maintain an experimental, innovative attitude 
toward the entire area of testing and the educational issues con- 
nected with it. Perhaps the greatest danger is the temptation to 
focus exclusively on testing as the central problem area. It would 
seem that in reality the major problem areas arc the admissions 
policies of the member colleges and their curriculuins and meth- 
ods of instruction. To focus on tests in isolation, without regard 
to these two areas, would seem to offer little hope of improve- 
ment. 

The central theme of the Commission on Tests’ discussions, 
and the one that seems most nearly to command full agreement 
of the Commission members, is the principle of “symmetry”— 
the mutual availability of relevant information to all parties in a 
system of reciprocal selection. The choice among colleges by stu- 
dents, and the choice among applicants by colleges constitute 
such a system. In economic terms this is a market situation; the 
market will be more nearly perfect, and the benefit of the trans- 
action to all parties maximized if the fullest possible information 
bearing upon choices is made available. 

This central theme is developed i:i briefs by other members 
of the Commission. My purpose in this one is to point out why I 
believe the existing test mode to be less than satisfactory in the 
context of admission to continuing education, and why other 
approaches to the problem of assessment should be developed. In 
a separate brief, I have described in some detail one such alterna- 
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tivc mode: self-scored and sclf-adminrtcred tests (pages 146- 
160). The present brief will have fulfilled some of its purpose if it 
can help to introduce into the minds of educators and of the pub- 
lic the realization that there can he many other ways of carrying 
out the broad function of educational assessment besides the cur- 
rent almost universal employment of adversarial, normative, 
psychometrically oriented, mass-administered, secure, speeded, 
multiple-choice, pcncil-anil-paper tests. 

Such tests, now in wide use i:i this country, have proved to 
be a useful device during a particular state of the nation's educa- 
tional development. I do not condemn them out of hand, cate- 
gorically, as do some critics. But I see aspects of the testing struc- 
ture that seem not to fit the problems of the present and the 
visible future. I sec great danger in complacently continuing the 
existing system and defending it uncritically. I sec, in particular, 
ways in which the general ambiance of the testing process, in- 
cluding the ways in which colleges utilize tests in their admis- 
sions processes, works harm to the student in the sensitive as- 
pects of his own self-confidence, sense of identity, and the 
energy of spirit without which the cognitive aspects of learning 
become shriveled and lifeless. The Board ought to be profiting 
from criticism and exploring possible improvements on a wide 
front. 

It is obvious that the structure of tests and the practices of 
testing constitute a prime example of the general sociological 
principle that a new technology, innocently devised to meet a felt 
need, may, because of its very convenience and utility, become 
an uncomfortably rigid mold that comes to constrain and limit 
the people whose convenience it was designed to serve. The 
automobile is of course the prime example of a technical device 
that, because of its obvious utility, has spread enormously. Not 
only has it disrupted cities, but it has developed secondary and 
tertiary ill effects that at the outset seemed negligible. Among 
them are accidents, air pollution, noise, and congestion. In much 
die same way the apparent utility and convenience of testing in 
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the current mode brings serious consequences as this style be- 
comes so widespread as to dominate curriculums and teaching 
methods, and to distort students’ concepts of the structure of 
knowledge and the nature of intellectual achievement. 

The automobile, like the standardized test, took its origin 
from a simple technical innovation developed in Europe around 
the turn of the century. Both devices owe their luxuriant growth 
more to the American genius for mass production and marketing 
than to any notable originality exhibited in their development 
since they reached these shores. The automobile has been char- 
acterized as an engineering success but a scientific failure. Some- 
thing of the same dichotomy of attributes is manifest in the 
standardized test. Granted the initial premises on which it is 
based, its development is a triumph of logical and mathematical 
perfection. But not enough is known about the learning process, 
the structure of knowledge, or the nature of cognition to support 
any confidence that the testing process yields more than a super- 
ficial, empirical body of data. 

The psychometric basis of testing seems to erect a formidable 
structure of mathematical logic on a foundation of psychological 
assumptions that are of uncertain validity. The logic is irrefut - 
able, but little is known about the underlying psychology. I am 
not convinced that we have got tully to the bottom of the philo- 
sophical question of what we arc measuring. A number of critics 
have pointed out that current tests measure not what the stu- 
dent knows but what he docs not know. But since what each of 
us does not know is for practical purposes infinite in extent, test- 
ing practice presupposes a theory of cognition based on the as- 
sumption that there exists a limited, definable corpus of “knowl- 
edge.” Even tests carefully designed to measure intellectual skills 
rather than factual knowledge (and the Board has increased the 
proportion of these) presuppose a similar limitation. They sim- 
ply identify gaps in a structure of “knowledge” made up of facts, 
theories, habits, attitudes, and systems of symbols. Many critics 
are disturbed at this situation, considering how much some stu- 
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dents know and can do beyond the predetermined corpus of 
knowledge. One wonders how much of the corpus itself is really 
essential, seeing how much is being dropped out of it as well as 
added to it year by year, and how arbitrary much of the content 
is. Old curriculums quickly begin to look quaint, and outmoded, 
and today’s students readily adapt themselves to the world 
around them without the armament of “essential knowledge” 
held indispensable only a few years ago. Beyond very elementary 
common learnings the concept looks increasingly dubious. 

We in education have already learned to cringe when people 
talk about “intelligence tests.” One wonders whether a cultural 
anthropologist would not cringe even more at our easy use of 
the term “aptitude.” He might marvel at our arrogance in as- 
signing any normative or absolute value to the symbol systems 
of a particular culture and its traditional habits of thought. This 
is not a fanciful idea, but a pressing practical problem. Large 
minority groups have rich, well-established cultures of their own 
—cultures with a high survival value in coping with difficult 
environments. We have to ask ourselves whether it is not we in 
the academic world who are “culturally deprived.” If higher 
education ecu Id and would reach inside these cultures and under- 
stand them there might be less arrogant insistence that others 
must come the whole way toward particular “correct” habits of 
thought. 

Not only contemporary test theory, but contemporary 
higher learning in general leaves little scope for Josh Billings’ in- 
sight that “we are all ignorant, but of different things.” The aca- 
demic world tends to say “If anything’s known, we know it, 
and what we don’t knew isn’t knowledge.” 

Knowing as little as we do about the structure of knowledge 
and the very nature of cognition, we are entitled to be very skep- 
tical about scores derived by psychometric methods. It is certain 
that we need to know much more about the deep linkages be- 
tween cognitive processes and the emotional life of the indivi- 
dual before we can use such numbers with any confidence. The 
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bedrock reliance of psychometry upon validity studies is put in 
doubt if the criterion against which validation is carried out is 
itself dubious. This would be the case if the criterion turns out 
to be formal, traditional, arbitrary, or irrelevant, or involves a 
circular reasoning process because it resembles the test itself. 

So we need to take seriously the increasing chorus of public 
criticism about tests and testing. It is true that part of the conflict 
represents power struggles about which groups shall control test- 
ing, rather than issues of educational philosophy; some of the 
critics are intemperate and extreme; some are far from disinter- 
ested and have been accused of making a career of criticism; and 
some blame tests for abuses more properly traceable to the mis- 
use of tests, and to a misunderstanding of their nature and pur- 
pose. Yet the question persists whether a device that can be so 
badly misused and so widely misunderstood is really as good as 
it ought to be. In any event, the Board cannot afford to take a 
merely defensive posture, or content itself with simply pointing 
out the flaws in the critics’ arguments. 

The younger generation frequently perceives education as 
irrelevant, threatening, manipulative, and coercive. There is rea- 
son to believe that testing, as actually carried on, particularly in 
the context of admission to higher education, contributes to this 
impression. Essential to the psychological health of the individ- 
ual in any society is a feeling of identification, the sense of being 
a part of what is happening around him. This sense is best 
achieved when he feels he has a chance to say something about 
the circumstances and constraints that hedge him round, and to 
have that something listened to. Tests, when actually felt as a 
means of manipulating or coercing the individual, directly con- 
trovert this sense. Testing has developed primarily in the in- 
terests of the colleges, not of students. 

Colleges and other institutions of continuing education have 
too easily assumed that students will docilely accept the results, 
and have too easily convinced themselves that whatever they 
themselves did was all for the good of the student. The student 
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feels himself the defendant in an adversary proceeding not in- 
tended to advance his interests. 

So there is a growing number of attacks on normative na- 
tional testing for purposes of college admission and grading. 
One basis of such attacks centers on the argument that there is 
too much emphasis on general academic ability, as judged by 
one or two major dimensions, and not enough attention given to 
specialized abilities that may also be of importance in society. 
Discriminatory comparisons in scholastic aptitude tests may set 
up a self-fulfilling prophecy; those students who do well in the 
aptitude tests also do well in the curriculum that is geared to the 
same habits of thought with which the tests concern themselves. 
It is common knowledge that there is a very uncertain relation 
between college grades and later social effectiveness in occupa- 
tions other than specialized professional and scientific areas. If 
the curriculum itself is only partially relevant to the student’s 
interests and objectives, and therefore provides a criterion of 
questionable value for test validation, screening, and selection, 
the resulting meritocracy becomes diverted from rational human 
purpose. 

The entire educational system has assumed that its built-in 
academic standards are “correct.” It has ignored the influx into 
society of other and different cultural elements and has not been 
zealous to devise newer and better educational strategies. In a 
pluralistic society, these ethnocentric and academocentric as- 
sumptions need to be greatly modified. The testing movement 
as dominated by the Board, however, has accentuated them. 

The Board has of necessity reflected the conservatism of its 
member colleges. The Commission on Tests should recommend 
that the Board reverse its position and take the lead in experi- 
menting with a wider range of assessment devices. In this time 
of crisis support from the Board’s member colleges for an ex- 
perimental and innovative attitude may be more easily forth- 
coming than in the past. 

To recognize adequately the interest of the individual will 
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require more than a change in tests. It will require a change in 
the habits and assumptions on which college admissions policy is 
based. When testing is focused primarily on the individual rather 
than on the convenience of the college, there is imposed on col- 
leges a greater obligation to take risks. Attrition is to be ex- 
pected rather than shunned. If no attrition occurs, the college is 
playing it too safe. Dropouts are by no means always or neces- 
sarily a bad thing. The unexamined assumption that the version 
of educational experience that a particular faculty has produced 
is a valid educational standard results in many students being 
tagged as “failures” simply because they do not respond to that 
particular pattern of stimuli. 

Nonnative testing has led to a series of closed feedback loops 
through the following steps: (i) validation of tests (along with 
other predictors) against college performance (as expressed for 
example in grade-point averages); (2) prediction of performance 
based on a regression formula derived from this validation; and 
(3) selection of entering students according to the formula, thus 
maximizing correlation between predictor and criterion. This is 
a tightly constrained system of reasoning that gives little incen- 
tive to change conventional faculty-oriented methods of presen- 
tation or assessment. And college faculties made up of individ- 
uals whose very life blood is critical reasoning and assessment 
have often been laggard in applying these edged tools to their 
own attitudes and capacities vis-^-vis students. If they have shed 
illumination, it has been in the form of laser beams focused on 
their special fields. 

The extraordinary performance of many dropouts in terms 
of ultimate social effectiveness casts serious doubt on this con- 
ventional sequence of validation, prediction, and selection. Fac- 
ulties tend to want the “best” students according to these nar- 
row criteria. There results a “collegiocentric” selection in which 
faculty preference and convenience often takes precedence over 
the student’s interest in his own long-range development and 
social effectiveness. The unregencratc professor who remains un- 
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touched by any vision about the broad social consequences of 
education is happiest when he can teach more and more to fewer 
and fewer and better and better students. Faculty opinion is the 
dominant force in the admissions practices of the stronger insti- 
tutions. The testing problem is so closely involved with admis- 
sions policies that the Board’s freedom of action is limited. The 
deep roots of inertia reach back into the colleges and into beliefs 
about student selection that may prove ill-founded. There is real 
question whether individual colleges (i) can select the students 
who in the broadest sense are the most promising, or (2) are, in 
the public interest, necessarily entitled to do so. Yet these two as- 
sumptions have been, from time immemorial, unquestioned 
dogma in most colleges. 

Skepticism about the closed circle of validation, prediction, 
and selection is justified not only on these bre 'd grounds but 
also more narrowly by increasing doubts about the entire struc- 
ture of marks and grade-point averages in higher education. 
These devices are being increasingly condemned as anti-intellec- 
tual in tendency. A device that began as a practical convenience 
in dealing with large classes becomes an end in itself, diverting 
the attention of both teacher and student from the central con- 
cerns of intellectual development. In response to these doubts, 
the structure of marks and course credits is being eroded by vari- 
ants of “pass-fail,” and similar arrangements. Marks are seen as 
“minimum information” devices. Their use as a criterion is thus 
open to question also on the practical ground that they will be 
less available as time goes on, as well as less closely coupled with 
the realities of what is happening in the mind of the student. 

The conventional quantitative approach to testing empha- 
sizes the status achieved after learning rather than an identifica- 
tion and understanding of the process and mechanism by which 
learning occurs. Testing, then, if it is to represent assessment in 
the broadest terms should determine the appropriate comple- 
mentation rather than comparison. The student should be edu- 
cated to complement the learning he already has, rather than 
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just to compare it with the learning others have. 

Monitoring of performance as a basis for determining qual- 
ity can be an on-going part of teaching rather than primarily a 
means of determining the destiny of the individual. 

There is a wide field still undeveloped for the characteriza- 
tion (not necessarily measurement) of attitude, function, motiva- 
tion, style, and temperament as well as of ability and achieve- 
ment, and ample scope for research in developing the kinds of 
instruments needed for such characterization. 

Beyond the individual college, testing is closely related to the 
wider issue of whether the annual distribution of entering stu- 
dents among some 2,000 colleges approaches an optimum pat- 
tern from the standpoint of the public interest. No one knows 
whether it does. Few individual colleges have even glimpsed the 
existence of this issue, so intent are they on their own supposed 
institutional interests. The biologist, J. T. Bonner, makes the 
arresting suggestion that the selective colleges may in fact be all 
unconsciously effecting a kind of counterselection that diverts 
the most promising students into the nonselective, “open door” 
colleges . 1 

Prediction and selection can never be wholly eliminated in 
education, but increasing emphasis should be laid rather on di- 
agnosis of the individual’s needs and on prescribing the next step 
he should undertake. Testing with this objective would exist in 
an affirmative context of optimizing the benefits of the next 
step rather than in an exclusionary context of deciding who is to 
be denied a specified educational opportunity. 

The traditional approaches to testing for college admissions 
are ill-adapted to the diagnostic objective. They are essential 
only if one grants as valid current college faculty preferences and 
habits and current admissions policies centered on the supposed 
interests of particular institutions in maintaining these prefer- 



i. “A Biologist Looks at Unnatural Selection.’* Princeton Alumni Weekly , Nov. 
23, 1962, pp. 6-16. 
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ences and habits. These are in part traditional and retained 
through sheer inertia, but in large part they are traceable to the 
preference of college teachers to teach those who arc easiest and 
most fun to teach. These generally turn out to be those already 
furthest along toward becoming copies of the teacher himself. 
It is these, in the jargon of the trade, who are said to be able to 
“profit” by higher education. 

Tests in the context of transition to college were originally 
devised to serve the interests, real or supposed, of the colleges 
themselves. The basic assumption was that a certain minimum of 
formal knowledge was indispensable for the beginning college 
student. The present use of test scores in r. normative context 
introduces a competitive factor that was at most a minor part of 
the original plan. The intensity and artificiality of the competi- 
tive aspect in recent years has strengthened the antitest revolt. 
Even the College Board’s Preliminary Scholastic Aptitude Test, 
which was, nominally at least, devised to aid guidance, is used 
in practice mostly to adumbrate the probabilities of admission to 
particular colleges. 

Aptitude tests that measure mastery of the mother tongue of 
the dominant subculture impose a heavy and permanent handi- 
cap on all minority groups. There is a bitter truth in the oft-re- 
peated injunction that one “cannot study for” the College 
Board’s Scholastic Aptitude Test (sat). One has to be bom and 
raised to it. It is of interest that “disadvantaged” students gen- 
erally lag less in mathematical than in verbal skills; the quantita- 
tive symbol system is learned later and somewhat more artifi- 
cially than the verbal system of the mother tongue. To learn it 
requires a conscious effort on the part of all, whether or not they 
are already at home in the use of the mother tongue. Thus, in 
arithmetic, all start more nearly even. 

This issue about the “unfairness” of aptitude tests with refer- 
ence to particular minority groups is of course an old one. The 
conclusions of most discussions about it are (i) that since com- 
petence in standard English is indeed a prerequisite to any effcc- 
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tivc higher education, aptitude tests, however “unfair,” must 
continue to stand, and (2) that the remedy lies in eariy education 
—Head Start and all the rest of it. All this, up to a point, is true. 

But there may be a broader dimension to the problem. To 
anyone brought up in traditional academic disciplines the writ- 
ten expression of many high school students, even those in the 
main stream of the national culture, seems semiliterate and bar- 
barous. It is possible to function successfully and effectively in 
society using language of a kind to make any college professor 
shudder. So the possibility that there may be a cultural lag here 
that will never be made up must be entertained. Standard Eng- 
lish, sat version, may be an echo from the past of a generation 
or two back. If (as seems to be the case) the effective work of the 
world in the long run turns out to be carried on in large measure 
by people with a very wide range of test scores and widely vary- 
ing degrees of literacy, it may be that prim insistence on v ^se 
so-called predictors is not so much a necessity as an academic 
foible. 

There is an inevitable and everlasting tension, which testing 
may have to reflect, between the “official” language of the dom- 
inant few and the vigor of popular usage. In John Hcrsey’s elo- 
quent words, written in a letter 10 me: “The latter feeds into the 
former and keeps it warm. Testing must go on discovering the 
future guardians of precision in human communication; but test- 
ing must stop acting as if these were the only educable human 
beings. Let us not cave in to the motion toward the grunt as the 
only form of human speech; let us also, though, whisper into the 
cars of some, who by misfortune have only learned to grunt, 
the words, let’s say, of Shakespeare, who was white, of Lao Tze, 
who was yellow, of Aesop, who was black, of Pushkin, who 
was black-and-white, to show that precision belongs not to a 
‘dominant subculture’ but to mankind and that we all need it in 
order to live together. This is not to ‘lower standards.’ Elegance 
and precision in the use of words will be no less pursued than all 
other manifestations of excellence. But let us not debar from 
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educational opportunity those upon whom these ornaments 
have not descended as a birthright.” 

The Commission heard testimony from the examining com- 
mittees of the Board which led to the conclusion that the present 
Achievement Tests of the Board have been reduced almost to the 
condition of tests of academic aptitude just a little more special- 
ized than is the Scholastic Aptitude Test. This suggests that the 
Board as a conservative first step might aim at eventual phasing 
out of the Scholastic Aptitude Test in favor of a variety of diag- 
nostic instruments to measure particular skills and competencies. 
Tests like these would have the great advantage of increasing 
flexibility. The college could choose, or determine how far it 
would allow the student to choose, what assortment of abilities 
it would ask for. The rigidity of the sat with set parts of pre- 
determined size, would be reduced. 

The above recommendrtion is simply a practical change well 
within the conventions and philosophy of the existing system. 
Briefs by other Commission members, and one of my own, sug- 
gest a much wider range for exploration and experiment. 

The Board should explore various versions of student-cen- 
tered tests, both those aimed at answering the student’s questions 
about how he compares with others in conventional subject- 
matter areas, and those aimed at gauging his response to various 
educational styles and environments. Modular tests of compe- 
tency in clearly defined small units have great promise, as do 
aids to the student in organizing and utilizing his own self-esti- 
mates. Throughout, the measurement objective should so far as 
possible be subordinate to the imaginative grasp of the subject 
area itself. The response of the intellect to the world around it, 
the alternation of long and near focus, the emphasis on the one 
hand of contemplative thought and on the other of hair-trigger 
response, arc all values to be conserved and studied. They should 
weigh heavily as against supposed increments of precision in 
measurement. 

In sum the Board should probe, experiment, explore, and 



above all refuse to allow its collective imagination to be tethered 
to conventional modes. It may be difficult to move the now pon- 
derous Board-ETS complex out of its accustomed grooves, but 
much more difficult will be the task of getting college faculties 
to change their ways. The Board, if it wishes, can make its in- 
fluence felt in this direction. The colleges are worried and begin- 
ning to be willing to take a second look at themselves. 

A Note on the Speededness of Existing Tests 

It is my conviction that the speed clement in the current Board 
tests constitutes a more serious constraint on their usefulness than 
seems to be recognized in the ets studies of the subject. These 
studies conclude that the tests in general arc only “slightly 
speeded.” ets routinely produces extensive, detailed statistics on 
degrees of speededness among various tests. These are believable 
and useful as intertest measures of relative speededness. What they 
do not and cannot provide is any absolute measure of speeded- 
ness or its effects on the student's thought processes or his mo- 
rale. Speededness as a psychological principle is like entropy: one 
can measure increments, but never the absolute amount. 

The basic assumption, which seems to me fallacious and, on 
the face of it, erroneous is the following: “A test may be re- 
garded as essentially un speeded if at least 80 percent of the candi- 
dates reach the last item, and if virtually everyone reaches three-- 
quarters of the test .” 2 The various statistical matrices that show 
in an ingenious way the relative degrees of speededness of par- 
ticular tests are all erected on top of this basic assumption. The 
assumption seems erroneous, since it is unquestionably true that 
the great majority of candidates are under the threat of not finish- 
ing, and this includes most of those who actually do succeed in 



2. From an unpublished analysis of the January 1967 administration of the Col- 
lege Board Achievement Tests. Princeton, N.J.: Educational Testing Service, 
1967. 
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reaching all the items. Under this threat, the student’s entire ap- 
proach to each item is of necessity based on snap judgment, not 
on considered and deliberate thought. He knows that speed is of 
the essence, and that a lack of it is quite likely to affect his score 
adversely. This issue illustrates the general truth that the quanti- 
tative and statistical aspect of normative testing must rest, at 
last, on psychological assumptions that can only be tested in 
vague and subjective ways. The Board should prefer evidence 
of intellectual power to speed of response. 

If, as I hope, greater use comes to be made of student-cen- 
tered, self-scored, and self-administered tests, the speededness 
issue will be somewhat less serious. Meanwhile I believe the 
Board should consider some reduction in speededness by reduc- 
ing the number of test items, even at the cost of some slight loss 
of reliability. 

The speededness issue is related to the purely practical ques- 
tion of whether it is important that a normal test program be 
completed in a single day. When test centers were fewer, and 
transportation more difficult, the necessity of staying overnight 
near a test center imposed serious hardships on students. If this is 
still the case, it strengthens the argument for reducing the num- 
ber of tests in a day, with more time per test. This matter de- 
serves review in connection with the speededness issue. 

Recommendations 

It is recommended that the College Entrance Examination Board: 

1. Experiment actively with a system of modular, diagnostic 
instruments designed to test important competencies; these 
would, in principle, resemble the present Achievement Tests. 

2. Make these available to colleges as a means of permitting 
greater flexibility in the choice of combinations of tests selected 
by the student, and required or permitted by the college. 

3. Continue the Scholastic Aptitude Test for the time being, 
with such improvements as may be possible. 
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4- Encourage member colleges gradually to replace the 
Scholastic Aptitude Test with its fixed, predetermined parts or 
modules, by more flexible assortments of modular tests. 

5. Reduce the speededness of its tests by reducing the num- 
ber of items per test, even at the cost of some loss of reliability. 



Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
Diversification in Educational Assessment 

It is recommended that the College Entrance Examination Board: 

1. Experiment actively with a system of modular, diagnostic in- 
struments designed to test important competencies; these would, 
in principle, resemble the present Achievement Tests. 

Endorsed by 17 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Fricdenbcrg, Gordon, Hechingcr, Hcrscy, Holtzman, 
Hoy, Marland, Pearson, Prentice, Shirley, Thresher, Ticdcman. 

Carroll: “In particular I wish to express reservation about 
one aspect of Thresher’s first recommendation, namely that his 
modular, diagnostic instruments \ . . would, in principle, re- 
semble the present Achievement Tests.’ Thresher docs not de- 
velop this point in his brief; therefore, I cannot be sure in what 
respect he wants the proposed tests to resemble the present 
Achievement Tests. If he means that they should sample achieve- 
ments in various subject matters such as English, mathematics, 
science, and so forth, I believe they would be subject to many of 
the same criticisms that have been leveled against the present 
Achievement Tests— inadequate sampling, the assumption of 
uniformity in the curriculum, undue influence on the curricu- 
lum, improper timing of test administration in relation to the 
student’s stage of preparation, and so on. The whole experience 
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of the College Board has suggested that tests of intellectual skills, 
such as the parts of the Scholastic Aptitude Test (sat), are much 
less subject to this kind of criticism. If I were to rewrite this rec- 
ommendation, I would suggest that the last clause read as fol- 
lows: these would, in principle, resemble the present aptitude 
tests. They would resemble the present aptitude tests in that they 
would measure general intellectual skills of various sorts, rather 
than specific knowledge or skills in specific subject matters. 

“Thresher seems somewhat resistant to the idea of testing 
general intellectual skills. He claims that it is an affront to the in- 
dividual to be required to take a test that he cannot study for. I 
would argue that this claim reflects a very narrow view of what 
intellectual skill consists of. To be limited to constructing tests of 
achievements that an individual starting front scratch could study 
for in, say, no more than a few months would be very limiting 
indeed. Perhaps a test of a foreign language vocabulary or a test 
of knowledge of the geography of Rhode Island would be ex- 
amples of the kinds of tests that could be given under this re- 
striction. The fact is that the individual never starts from scratch 
(at least, the individuals likely to be under consideration in the 
college admissions process). The very act of studying for a test 
usually presupposes a long educational history; even studying 
for a test on Rhode Island’s geography presupposes that the stu- 
dent can read and understand the language in which his lessons 
are couched, can remember, can make inferences, and so forth. 
In testing general intellectual skills we are more interested in 
what the individual has acquired from his lifetime educational 
history than in what he may have acquired from a short term of 
study. 

“In a sense, a student does study for the present sat. That 
study, however, is not something he can do overnight or even 
in a few months; the study is what he docs in the course of his 
whole educational history. But it is a sheer exaggeration to say 
that one has to be born and raised to the culture implied by the 
sat in order to do well on it. Many people with ample educa- 
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tional advantages do not do well on it, but many people who 
would appear to have suffered educational disadvantages— poor 
formal education, bilingual background, or the like— have been 
able to do well on it, by virtue of their own efforts to read and 
educate themselves over a fairly protracted period.” 

Shirley : “I certainly endorse this recommendation to experi- 
ment and test this mode of operation against the present pro- 
gram. No one can be against research or experimentation on the 
effectiveness of what testing is truly doing. New testing modes 
should be sought, but simplification, rather than complication, 
of the testing process and program should be the touchstone.” 

Not endorsed by 1 membe ¥ ' Schlossberg. 

l member abstained : Pace. 

2. Make these available to colleges as a means of permitting 
greater flexibility in the choice of combinations of tests selected 
by the student, and required or permitted by the college. 

Endorsed by 16 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Hechinger, Hersey, Holtzman, 
Hoy, Mar land, Pearson, Prentice, Thresher, Tiedeman. 

Prctitice: “Security can be a serious problem if tests are made 
available to colleges. If that problem can be overcome, I approve 
of the recommendation.” 

Not endorsed by 2 members: Schlossberg, Shirley. 

Shirley: “This recommendation (and Recommendation 4) 
prejudges the results of the research requested in the first recom- 
mendation. The decision about what tests should be continued 
ought to be based on the research and exploration. If we now 
know what is proper, we should advocate elimination of the 
present tests and adoption of new ones. At this stage only one 
thing is sure— we do not know; we have only biases, prejudices, 
and beliefs.” 

1 member abstained: Pace. 
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3. Continue the Scholastic Aptitude Test for the time being, 
with such improvements as may be possible. 

Endorsed by 16 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Gordon, Hechinger, Hersey, Holtzman, Hoy, Marland, 
Pearson, Prentice, Shirley, Thresher, Tiedeman. 

Not endorsed by 2 members: Friedenberg, Schlossberg. 

i member abstained: Pace. 

4. Encourage member colleges gradually to replace the Scholas- 
tic Aptitude Test, with its fixed, predetermined parts or mod- 
ules, by more flexible assortments of modular tests. 

Endorsed by 16 members: Abelson, Blocker, Carroll, Coleman, 
Foster, Friedenberg, Gordon, Hechinger, Hersey, Holtzman, 
Hoy, Marland, Pearson, Prentice, Thresher, Tiedeman. 

Foster: “As is now being done by the Board.” 

Not endorsed by 2 members: Schlossberg, Shirley. 

Shirley: “See my comment under Recommendation 2.” 

1 member abstained: Pace. 

5. Reduce the speededness of its tests by reducing the number of 
items per test, even at the cost of some loss of reliability. 

Endorsed by 14 members: Blocker, Carroll, Coleman, Foster, Frie- 
denberg, Gordon, Hechinger, Hersey, Hoy, Marland, Pearson, 
Shirley, Thresher, Tiedeman. 

Foster: “Possibly, but this is taken care of in part by Recom- 
mendation 1, above.” 

Hersey: “Speed of response may well be one of the gifts that 
should be considered valuable for some but not all processes 
and functions and perhaps one mode of a set of diagnostic in- 
struments should test for it.” 

Not endorsed by 4 members: Abelson, Holtzman, Prentice, Schloss- 
berg. 



Abelsort: “In general I agree with Thresher’s comments and 
recommendations, except for Recommendation 5 on speeded- 
ness. A very important factor in a person’s ability to operate 
successfully in this world is to be able to function effectively 
under stress. Very frequently the ability to make quick decisions 
at a moment of crisis is a determining factor in success or failure. 
I feel that to comply with Thresher’s Recommendation 5 
would be a step backward.” 

Holtzman: “I would prefer allowing more time as a possible 
alternative to reducing the number of items— at least for some 
tests— since reliability is of importance in individual measure^ 
ment.” 

Prentice: “If tests do not predict something, they cannot be 
said with confidence to convey any information. If reliability is 
diminished, so will prediction be. Validity is very low now, and 
I do not think we can justify poorer prediction. The concern 
about speededness is based on the belief that some students have 
‘power’ not measured in speeded tests. The way to get at that 
‘power’ (if it exists) is to design tests that are fair to that minority 
of students. In principle, such tests should be more reliable and 
more valid.” 

1 member abstained: Pace. 

General Comments 

Carroll: “I endorse these recommendations, since they seem in 
general to agree with the recommendations in my brief, “Possi- 
ble Directions in which College Entrance Examination Board 
Tests of Abilities and Learning Capacities Might Be Developed,” 
but I wish to express reservations about some of Thresher’s as- 
sumptions and also about certain particulars of his recommenda- 
tions. 

“For the record, I must state that I cannot agree that the psy- 
chometric basis of testing seems to erect a formidable structure 
of mathematical logic on a foundation of psychological assump- 
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tions that are of uncertain validity. It would take me too long* to 
develop my point adequately, but I would affirm that present 
testing practice is based on a rather large body of theory and on 
empirical evidence that hangs together with that body of theory. 
To a large extent psychometricians do know what they are meas- 
uring. For example, factor analyac research has disclosed a num- 
ber of independent dimensions of cognitive ability, and it is pos- 
sible to construct fairly refined tests of each of these dimensions. 
One psychologist, J. P. Guilford, has developed a theory of the 
structure of the intellect that attempts to classify cognitive abili- 
ties according to a grand taxonomy based on the content and 
outcomes of mental operations. I would be the first to point out 
that knowledge about cognitive abilities is still imperfect and in- 
complete, and that there is not universal agreement about what 
is known; yet, I would also be the first to refute the impression 
conveyed by Thresher’s brief that testing practices are built on 
sand. 

“In the case of subject-matter achievement tests I can per- 
haps understand the criticism that current tests measure not what 
the student knows but what he does not know, but I do not 
think the criticism has much force when applied to tests of in- 
tellectual skills. In large measure, the intellectual skills that arc 
involved in higher education can be identified, and how well a 
student has acquired these skills can be measured with consider- 
able accuracy. Indeed, I personally attach more credibility to 
test outcomes than to the criteria that are normally used for vali- 
dating tests. Granting that validity studies smack of circular rea- 
soning, I hesitate to enter the circle. 

“I am willing to follow Thresher in feeling skeptical about 
the assumption that the intellectual skills measured with present 
tests are indeed the intellectual skills most imporant for academic 
or other kinds of success. This does not mean, however, that we 
who are concerned with testing should completely lay aside our 
present knowledge of cognitive abilities. Rather, we should 
work within the present framework of theory and established 
scientific procedures to perfect our knowledge and improve our 
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testing practices. If the educational system is changing its built-in 
academic standards (and I am not sure that the standards are 
changing as rapidly as Thresher implies), test research should 
take this into account. Test research also can and should take into 
account the fact that society is pluralistic and has many subcul- 
tures and different sets of standards and values. 

“In summary, I feel that the line of argumentation that 
Thresher adopts is very seductive and very much in tune with 
contemporary rhetoric in some quarters, but it fails to take ade- 
quate account of scientific knowledge and perspective in the 
psychology of individual differences and learning.” 

Foster: “In Thresher’s first paragraph he says the College 
Board should be prepared gradually to reduce reliance on pres- 
ent testing modes. I think the phrase ‘if it proves advisable’ 
should be added (we don’t know yet). 

“Thresher states that testing has developed primarily in the 
interests of the colleges, not of students. I am still old-fashioned 
enough to believe tests are a useful educational experience and 
can even be enjoyable when not taken as life-and-death matters.” 
Pace: “I see this brief as a general essay— of substantial merit 
—on some of the educational, philosophical, and psychometric 
assumptions upon which much of present testing practice rests. 
As such, it is a significant background paper for subsequently 
explaining why changes in certain directions are recommended. 
I do not see that the particular recommendations are a necessary 
conclusion from the brief. Therefore, I choose not to endorse any 
of them specifically. Rather, I thoroughly endorse the intent and 
reasonableness and need for what is implied by the title of the 
brief— that is, diversity in educational assessment.” 

Schlossberg : “The reason I do not endorse these recommen- 
dations is that I think competencies can be assessed in other 
than paper-and-pencil ways. Games, simulation, work samples, 
and so forth would diversify measurement procedures, but 
just increasing kinds of achievement-type tests is not the 
answer.” 
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The principle of symmetry that forms the central theme of the 
Commission’s recommendations can be implemented in a num- 
ber of ways, all calculated to increase the student’s knowledge 
and understanding of himself, the educational opportunities open 
to him, and the world around him. The present brief suggests a 
single mode that seems promising. It is not proposed that this re- 
place all other kinds of testing, but rather that it be initiated and 
experimented with seriously with the idea that it might come 
to form an important part of the Board’s armamentarium of 
tests, in this mode psychometry would be less central to the 
operation than it now is. The psychologist and statistician would 
remain important auxiliaries, but the teacher would be the cen- 
tral and determining influence to a much greater extent than at 
present. 

Every teacher knows that the best time to get an idea across 
is right after students have taken a test on the subject and are 
keyed up, alert, and questioning. A discussion at this juncture 
can be unusually fruitful because it falls on prepared ground. It 
is unsound not to utilize this important principle. I think it is 
possible to go further and say it is demeaning, unethical, and 
educationally unsound to subject a student to a test without dis- 
cussing it with him, or if this is not feasible, at least providing 
him with a reasoned analysis of it as soon as possible afterwards. 
If no discussion takes place, the student reads the signal; they arc 
more interested in judging me than in educating me. This lack 
of education follow-through constitutes one of the major draw- 
backs of the present mode of testing. Present tests not only miss 
entirely the opportunity to extend and deepen understanding of 
the particular subject being tested. They have the more serious 
defect of failing to reinforce in the student the habit of question- 
ing, discussing, developing ramifications of thought, comparing 
frames of reference, viewing in more than one perspective. In 
general there is a complete failure to open out the subject by any 
play of the intellect. The habit of dropping all further concern 
with a topic once the “right” answer is found is damaging to the 



life of the mind. The student never comes within sight of the 
idea that no subject is ever permanently closed off from discus- 
sion, no conclusion forever final and irreversible. It is this anti- 
intellectual effect of the prevailing test mode that constitutes the 
most serious defect in a system that otherwise has much to com- 
mend it. Measurement is not without its uses, but it has come to 
crowd out other and more subtle but more important objectives. 

Proposals to meet this situation have included branching 
tests and tests that seek to determine the student's reasons for 
having chosen certain answers to a previous test. These have 
some merit, but neither meets the central problem because no 
reasoned analysis ensues. 

The central proposal here, therefore, is that self-administered 
and selfocored tests be instituted and made available to students 
in great profusion. I believe Coleman proposed this principle at 
the first meeting of the Commission on Tests, and the present 
proposal simply elaborates one phase of his idea. 

The testimony heard by the Commission from teachers on 
test committees was most illuminating. They were doing a 
skilled job constructing items with real teaching merit. The more 
they followed their teaching impulse, the more frustrated they 
became, because there was no opportunity for the student to 
benefit from this devoted effort. Only a tiny residue of “meas- 
urement’' came out of the whole process. There is a kind of 
tragic irony in thus setting good teachers to do a job that throws 
away most of their talent and skill. One could tell from their 
remarks that they sensed it. 

Under the present proposal, every time a test committee 
generates a test item, they would produce an accompanying 
reasoned discussion about the choice of an answer. This would 
go beyond the simple justification for the “right" answer as now 
given for the illustrative questions in the explanatory booklets 
prepared for the tests. Possible reasons for choosing alternative 
answers would be discussed, taking account of how a student 
with a different frame of reference might approach the question. 



147 



B. Aldcn Thresher 



Some item analysis and pretesting statistics might be included. 
In other words what is now ccmpulsively and secretively cov- 
ered up would be thrown open, and the student invited to par- 
ticipate in the exercise of reason. Both test and statistics would 
be put in the public domain. 

Depending on the use and purpose of the test, the student’s 
access to the reasoned explanation could be deferred by varying 
degrees. Two principal cases suggest themselves: (i) For truly 
self-administered and self-scored tests, the discussion could be 
put at the back of the test booklet, either with a seal or with un- 
cut pages. The student, if he really wishes a self-appraisal could 
be told to read this material, preferably as soon as possible after 
finishing the test. (2) For truly “adversary” testing in the present 
mode, with measurement the prime purpose, the discussion 
could be distributed to testees at the end of the test, or even sent 
to them later. 

Advantages of the Student-Centered Approach 

1. The problem of speededness vanishes in a self-scored test ex- 
cept as retained at the student’s option. It could and should be 
moderated in any tests that continue to be conducted in the com- 
pletely adversary mode with the student as defendant. 

2. In a self-scored test the security problem vanishes. If the 
student wishes to cheat, he cheats only himself and would proba- 
bly also cheat at solitaire. 

3. The student would no longer feel threatened or manipu- 
lated, or be a defendant in a process carried on not necessarily 
for his benefit. Even in the adversary version, he would not be 
flying completely blind. 

4. The test committees could at last feel that they arc func- 
tioning as teachers. Especially impressive was the testimony 
heard by the Commission from the history and physics people, 
who seemed to me to be suffering from frustration as good 
teachers not allowed to teach. A score is, after all, a minimum- 
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information device, and critics are undoubtedly right in object- 
ing that this is not an adequate summary of any student, even 
within the limited area that the test purports to measure. 

5. Self-tests and quizzes have an immense psychological ap- 
peal as is apparent from the number constantly appearing in the 
popular press both in serious and frivolous contexts. To test 
oneself answers a deep human need. It helps the individual an- 
swer the, to him, all-important question: “How am I doing?’' 

6. Face validity or curricular validity would be restored as 
a major criterion for test items, and a corresponding reduction 
could occur in the kind of statistical pretesting and time analysis 
now carried on. Along with this would go a reduction in the 
emphasis on precise measurement, or rather the pretense of it. 
Granted that evaluation has its place, it cannot be, as applied to 
the individual and his prospects, more than a very rough guide. 
Marianne Moore has put it in a nutshell: “Why dissect destiny 
with instruments more highly specialized than components of 
destiny itself?” Psychometrics has gained an importance that 
should go rather to the processes of teaching and learning, not 
simply to their results. 

7. Finally, the proposal for self-scored tests recognizes the 
basic principle that the best way to encourage the student to be- 
come a responsible person intellectually is to treat him as if he 
were one, not to fence him with safeguards that imply that he is 
expected to try to beat the game. 

It will be objected that the proposal to accompany tests with 
a (printed) discussion of the question falls far short of the ideal 
give and take of oral, face-to-face discussion. This is a valid ob- 
jection, yet the proposal at least represents a vast improvement 
over the present system in which the student drops his answers 
into the void and they arc never heard from again— a situation 
nicely calculated to stifle intellectual curiosity. It would be well 
worth while to experiment on a small scale with groups meeting 
for oral discussion within a day or two after a test. Face-to-face 
discussion would give an opportunity to gauge the true psycho- 
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logical and intellectual atmosphere in which the tests are en- 
veloped, and would, I think, introduce a wholesome nonpsycho- 
metric element. The student, instead of being an “it,” to whom 
an instrument is “administered” could be looked over by an ex- 
perienced teacher alert to the psychodynamics of the group, and 
the student (probably to his amazement) could respond, react, 
answer back, and have his views taken seriously. 

Obstacles and Problems 

i. By far the biggest obstacles to this proposal would be a com- 
plete reversal in the viewpoint and habits of thought character- 
istic of all testing organizations. These, in their whole origin and 
purpose are psychomctrically oriented, instead of being pri- 
marily oriented to the teaching and intellectual values in tests. I 
would hope that the Commission on Tests might, with the co- 
operation of Educational Testing Service (ets), set in motion a 
searching study of this problem to see how far it might be made 
practicable to carry out changes of the kind suggested. It will in- 
evitably require a fresh initiative in a direction quite different 
from the present. Most important of all, it will bring into con- 
sideration a range of intellectual values that by their very nature 
cannot be made to stand or fall by the conventional “validity 
study” techniques. They will have to be appraised, in the end, 
by a global judgment in which teachers and students participate. 

2 . It would be necessary to increase the output of test items 
enormously. I recall being shown some years ago a file at ets 
containing something of the order of 100,000 reserve items for 
the Scholastic Aptitude Test (sat). These presumably have not in 
most eases been pretested or subjected to item analysis, or even 
to committee consideration. The policy has been to keep under 
lock and key a limited bank of test items carefully screened and 
pretested, and to economize by reusing them at subsequent test 
administrations. This miserly and secretive policy treats knowl- 
edge as if it were a closed and secret matter, hidden from the 
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public eye in the custody of a hieratic group. Nothing could be 
further from the spirit of free inquiry and universal publication. 

I propose doing just the opposite. The object would be to 
flood the market with more test items than either student or 
coaching schools could possibly deal with individually, so that 
it would be less trouble to study the subject than to coach or 
memorize individual items. These would be in the public do- 
main, protected by their great numbers. This ought to go far to 
put the coaching schools out of business. 

To produce test items on the scale I am contemplating would 
require the services of a much larger number of experienced 
teachers and a complete restructuring of the present group of 
small committees, who by their own testimony to the Commis- 
sion arc hard put to produce even the items now needed for 
various dates and administrations. I would see this as a unique 
and priceless opportunity to involve actively many more teachers 
in the testing process, which should be the natural complement 
to teaching. The College Board has been plagued for years with 
the recurring criticism that active teachers do not feel they have 
any genuine part in the testing operation. The Board with all its 
works remains to almost all secondary school teachers a remote, 
inapproachable power whose actions arc past understanding, and 
usually arbitrary and unreasonable. One can picture a new situa- 
tion in which teachers in general arc invited to contribute items, 
subject to screening by an expanded committee. A history 
teacher could say to a class: “Here arc some questions I have 
proposed to the College Board Committee. Do you want to dis- 
cuss them? Perhaps one will turn up on the test.” The proposal 
would bring Educational Testing Service a step nearer to becom- 
ing Educational Testing Service. 

Pretesting and statistical item analysis would presumably still 
be needed, but on a reduced scale especially for the fully self-ad- 
ministered and self-scored tests. Greater participation by teachers 
would go along with greater emphasis on face validity, and a 
reduction of the homogenizing tendency of statistical item anal- 
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ysis. Measurement might be crude, but with discussion and 
reasoned analysis playing a part, educational ends would be 
better served. Even “adversary” testing might well be affected 
by this trend. It might well be possible to drop from Board test 
scores the third digit, which has always misled the public by a 
spurious pretense at precision. 

3. Introducing self-scored tests would, I think, lead to an 
eventual phasing out of the College Board sat, as contrary in 
principle to the teaching objective. A test that one cannot study 
for is an affront to the student in appearing to brand him with 
an evaluation that no effort of his can hope to change. The even- 
tual phasing out of the sat would leave a body of tests along the 
lines of the College Board’s Achievement Tests now in use, per- 
haps with added subjects, perhaps with smaller and more nu- 
merous modules, and with considerable choice open to the can- 
didate. These would continue to serve such member colleges as 
wished to make use of them. It is essential that no Board mem- 
ber college feel that anything is being forced on it unwillingly, 
and the sat would presumably continue for some time, unless a 
general strike against it by minority-group students should put it 
out of business. 

The ets view is that the Achievement Tests have, over the 
years, come to be more and more like aptitude tests because 
there are more items on the methodology of each subject and 
fewer factual items. This tendency exists, yet to draw this con- 
clusion from it puts the cart before the horse. There will be in- 
creasing need in the future for people competent in data man- 
agement and in the art of learning how to learn, people who re- 
tain plasticity and mental flexibility. These qualities, methodo- 
logical though they may be, will increasingly constitute the sub- 
ject matter of education. Aptitude tests should be reduced to 
achievement tests in the sense of reporting mastery of profitable 
knowledge and skills. For example, the Achievement Test in 
English Composition and the verbal sections of the sat report 
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simply a mastery of standard English and have a high intercorre- 
lation. 

The basic objective should be to get the student’s intellect 
working and to make it second nature to him to use his intellect. 
To get a precise measurement is by comparison of secondary 
importance. A rough quantitative appraisal may actually be bet- 
ter because it is not yet known what qualities might be usefully 
quantified, or how much of the apparent precision in test scores 
is, as has been frequently alleged, spurious or premature. 

4, Any proposal to phase out the sat, the mainstay of the 
Board’s income, must raise serious financial questions. Under the 
general approach proposed above, there would seem to be three 
mitigating factors: (a) Phasing out would be gradual, on the as- 
sumption that some member colleges would want to continue 
the sat, for a time at least, (b) Then, if the sat were eventually 
dropped, member colleges who might wish to continue “adver- 
sary” testing (and this number would probably be large for some 
time) could make a greater use of the new Achievement Tests. 
This use would go far to replace sat revenues lost, (c) A more 
difficult problem would be maintaining a flow of the self-scored 
and self-administered tests to students and of financing them. 
The most obvious device would be to make the purchase of one 
or more of these a prerequisite to taking an Achievement Test 
of the adversary type. 

It would be premature to go into detail on these proposals 
at this stage. I believe that it is desirable, however, for the Col- 
lege Board to take a careful look at the broad principle involved: 
an increase in emphasis on the educational and intellectual aspects 
of testing, partly to supplement, partly to replace the exclusively 
measurement emphasis. Even if there should seem to be a good 
case for continuing tests as adverse procedures, the educational 
aspect could still be concurrently developed. Enlightened ad- 
missions policies in the colleges will be a prime requisite, and the 
Board can take the lead in encouraging them. 
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Self-scored and self-administered tests would ease the over- 
emphasis on test scores as an aid to guidance. Though guidance 
will, for a long time, require the dubious strategems of college 
choicemanship, it can be given also a needed impulse in the truly 
educational dimension. The student can begin to see that the im- 
portant thing is not where he goes to college, but what he does 
when he gets there. He can begin to develop some critical sense 
about his own efforts. He can test himself or, if he wishes to re- 
tain this option, have himself tested in a manner that engages, 
respects, and develops his growing intellectual powers. This is 
an introduction to the life of reason, as opposed to an attitude of 
“find out what they want and give it to them.” I hope very 
much that it may be possible to work out some device that will 
bring some of this spirit into the Board’s operations and even 
into testing at large. I offer the above as a first, groping effort in 
this direction. 

Recommendations 

It is recommended that the College Entrance Examination Board; 

1. Supplement its present programs with self-administered 
and self-scored tests, made available to students in great profu- 
sion, and accompanied, under various degrees of delayed avail- 
ability, by reasoned discussions about the choice of an answer for 
each item. 

2. Draw teachers, particularly those in secondary schools, 
into large-scale participation in the production both of these 
tests, and of tests in the conventional “adversary” mode. 

3. Stress face cr curricular validity in the selection of test 
items, stress items of teaching merit. 

4. Invite, experimentally at least, groups of students who 
have taken a test soon thereafter to participate in an oral discus- 
sion of the test under the Board’s auspices. 
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Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
A Proposal for Self-Scored, Self-Administered Tests 

It is recommended that the College Entrance Examination Board: 

1. Supplement its present programs with self-administered and 
self-scored tests, made available to students in great profusion 
and accompanied, under various degrees of delayed availability, 
by reasoned discussions about the choice of an answer for each 
item. 

Endorsed by 15 members: Abelson, Blocker, Coleman, Foster, 
Friedenberg, Gordon, Hersey, Holtzman, Hoy, Marland, Pren- 
tice, Schlossberg, Shirley, Thresher, Tiedeman. 

Holtzman : “My endorsement is qualified, since Tm not sure 
that we should really flood the world with such items.” 

Prentice: “It would be impossible, I think, for educators to 
disagree with Thresher’s motives and implied criticisms of pres- 
ent procedures, but I think there is a romantic appeal to his pro- 
posals. Only if the schools are doing an extraordinarily bad job 
of teaching should the College Board have to carry out Recom- 
mendation 1. If the schools really need such a national program, 
I favor the Board's trying to provide it; otherwise, the teaching 
value provided by a few hours of tests seems to me trivial com- 
pared with that provided in the 12 years that teachers have spent 
with every student.’’ 

Not endorsed by 4 members: Carroll, Hechinger, Pace, Pearson. 

2. Draw teachers, particularly those in secondary schools, into 
large-scale participation in the production both of these tests and 
of the tests in the conventional “adversary” mode. 

Endorsed by 14 members: Abelson, Blocker, Coleman, Foster, 
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Gordon, Hersey, Holtzman, Hoy, Marland, Prentice, Schloss- 
berg, Shirley, Thresher, Tiedeman. 

Holtzman: “Again, I have reservations about the wisdom of 
implementing this recommendation on a large scale.” 

Prentice: “If we accept Recommendation i, Recommenda- 
tion 2 is inevitable.” 

Shirley: “I understand that this is current practice.” 

Not endorsed by 3 members: Carroll, Friedenberg, Hechinger, 
Pace, Pearson. 

3. Stress face or curricular validity in the selection of test items; 
stress items of teaching merit. 

Endorsed by 16 members: Abelson, Blocker, Coleman, Foster, 
Friedenberg, Gordon, Hersey, Holtzman, Hoy, Marland, Pace, 
Prentice, Schlossberg, Shirley, Thresher, Tiedeman. 

Pace: “Presumably this applies to ‘achievement’ tests and is 
being done now as far as curricular validity is concerned.” 

Prentice: “This has great merit as long as ‘face validity’ does 
not come to replace predictive validity. Users of tests need both 
(and so do students!).” 

Not endorsed by 3 members: Carroll, Hechinger, Pearson. 

4. Invite, experimentally at least, groups of students who have 
taken a test to participate soon thereafter in an oral discussion of 
the test under the Board’s auspices. 

Endorsed by \ 6 members: Abelson, Blocker, Carroll, Coleman, Fos- 
ter, Friedenberg, Gordon, Hersey, Holtzman, Hoy, Marland, 
Pace, Schlossberg, Shirley, Thresher, Tiedeman. 

Carroll: “I endorse Thresher’s fourth recommendation if it is 
taken to apply only to tests now used, or likely to be used, in ad- 
missions, placement, and guidance programs, only because prob- 
ing students’ reactions to these tests would further help the 
Board understand the nature and functioning of these tests and 



156 



perhaps improve them in certain directions.” 

Holtzman: “OK as an experiment, on a small scale.” 

Shirley: “I see no reason not to try this, but I confess I am 
skeptical about what it will contribute. I was once a student, and 
it is my sincere conviction that I am wiser about education on 
the basis of my present experience than I was during my late 
teens and early twenties.” 

Not endorsed by 3 members: Hechinger, Pearson, Prentice. 

Prentice : “This strikes me as carrying the implications of 
Recommendation 1 to wholly impractical lengths.” 

General Comments 

Abelson: “Mr. Thresher’s remarks would be valid if al! examina- 
tions that students undergo were conducted so that there were 
no post-mortem discussions of the tests. I see no harm whatso- 
ever when a limited percent of the tests are conducted in the 
manner of the sat evaluations.” 

Carroll: “I am sympathetic with many of Thresher’s con- 
cerns, but I can endorse only one of his recommendations, and 
that only in a limited way. 

“The basic recommendation, to provide self-administered 
tests (presumably, in the main subject-matter achievement tests) 
for use by students and teachers in discussions of test content, 
would be very expensive to implement, and I doubt that it 
would achieve the objectives implied by Thresher in introducing 
his proposals. Further, there are some implications of the pro- 
posals that would take the College Board out of its proper 
sphere. 

“I agree that tests cannot have a truly educative function un- 
less they are followed up by discussion or other kinds of informa- 
tional feedback. Teachers have been advised to use tests in this 
way ever since the testing movement got started. Even Socrates 
used questions to stimulate thinking. But just how many teachers 
use formal or informal tests to stimulate discussion of test con- 
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tent and how much time they allocate to such activities are ques- 
tions for which there are no good answers. I have the impression 
that teachers do not use tests (even their own tests) in this way 
very extensively, for lack of time or inclination. There is too 
much to cover in the usual course of instruction. Even if self- 
administered, self-scored tests were made available to students in 
great profusion, I doubt that these tests could adequately sample 
curricular content or that they would be used as much as they 
would have to be to justify the cost of their construction and 
distribution. (Exactly who would pay for them, anyway?) Al- 
ready there are tests and books of test items on the market— some 
of them good, some not so good. I cannot sec why the College 
Board should get into that market. And I doubt that the avail- 
ability of the self-administered tests would really achieve the ob- 
jective of yielding symmetry in the relationship between the 
College Board (or the colleges) and the students. Students arc 
not likely to perceive the tests as being related to their chances of 
college admission. 

“In this context it is necessary to distinguish between the 
educative and the educational uses of tests. Tests arc used cduca- 
tively in the course of instruction. But tests can also be used edn- 
caticnally — in admissions, placement, guidance, and so forth- -or, 
in other words, in connection with various educational functions 
that may have nothing directly to do with teaching the partic- 
ular content of the tests. The traditional (and, in my opinion, the 
proper) role of the College Board has had to do with these latter 
functions. I do not believe that the Board should involve itself so 
intimately with the educative process as to circulate materials for 
direct use in instruction (apart from their use in connection with 
present Board programs). This is, in effect, what Thresher is 
proposing. Already the Board exerts too much influence on the 
content of the curriculum; Thresher’s tests would tend to in- 
crease this influence. 

“I sympathize with the observation that teachers who con- 
struct tests on College Board committees may feel frustrated by 



the fact that the educative effects of such activities are minimal. 

I would argue, however, that the educational effects of these ac- 
tivities are very far-reaching already, in the curricular influence 
of the tests. I would agree that if possible there should be a 
broader participation of teachers in the construction of the Col- 
lege Board tests; the present committees are in many cases too 
small and inadequately representative. But any proposal where- 
by teachers would participate in the con^ruction of tests for 
purely instructional purposes would be, in my opinion, out of 
the realm of proper activities for the Board.” 

Coleman: “The following comments apply to the first three 
recommendations. 

“A change in the general orientation of achievement tests 
toward face and curricular validity and toward a serious intel- 
lectual endeavor is, I think, highly desirable. The achievement 
tests would come to have greater similarity to the British and 
European examinations at the end of secondary school, and less 
similarity to aptitude tests, than is currently the case. It should be 
recognized, however, that this does make the test score more 
dependent on the quality of the applicant’s secondary school 
training than at present, and less dependent on his ability. It 
should be recognized also that some economical means of test 
construction and standardization or norming is necessary to 
combine the virtues discussed in the brief with the statistical vir- 
tues for predictive purposes that current tests have, without in- 
creasing the cost of testing to the student. 

“I believe it is possible now to devise testing procedures with 
the serious intellectual content and educative properties pro- 
posed in the brief, without forsaking the psychometric predic- 
tive properties of current College Board tests. Efforts should, I 
believe, be directed toward such a combination, rather than to- 
ward either refinement of present techniques in the direction of 
better predictions or tests with educative value but with poor 
predictive value. 

“It should be pointed out, in any discussion of timed tests. 



that curtailment of time in test-taking is a tester’s substitute for 
presenting a greater intellectual challenge in the content of the 
test, and as such is a rather poor substitute, whether used by a 
teacher or in a standardized test.” 

Foster: “Thresher’s statement in the second paragraph that 
to subject a student to a test without discussing it with him is 
‘unethical’ appears to me exaggerated. Granted it would be pref- 
erable to discuss each question with each student, the student’s 
interest in showing his competence must make the imperfect 
mass process worthwhile. Printed answers available for discu- 
sion in school would help, possibly assisted by a carbon copy of 
his responses kept by the student. 

“In the list of advantages of a student-centered approach I 
disagree with the second. Much as one believes in an ‘honor sys- 
tem’ where it exists, this suggestion may impair the accuracy of 
the information transmitted to the college. The student cheats 
more than himself. 

“About the eventual phasing out of the sat: only along the 
lines of substitutions suggested in Commission meetings and in 
fact already partly in effect. Later, in the same paragraph, I won- 
der if mentioning the possibility of a strike against the SAT by 
minority-group students is wise. 

“I question the wisdom of including the subject of the finan- 
cial questions involved in phasing out the sat. It puts the idea of 
bias from ‘personal interests* of the Board into the report and 
into the heads of many who would other wise not worry about 
it.” 

Pace: “One recommendation in the brief is not listed among 
the numbered recommendations — namely, that the sat be 
phased out gradually. I do not endorse this notion. Instead, I en- 
dorse Carroll’s proposals to revise the sat along certain more 
diagnostic lines.” 

Pearson: “I don’t believe these recommendations will ac- 
complish the objectives in Thresher’s brief.” 
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Can a Machine Admit an Applicant 
to Continuing Education? 

David V . Tiedeman 
Prologue 

My service on the Commission on Tests coincides with an op- 
portunity [ have had to construct and experiment with an In- 
formation System for Vocational Decisions (isvd) in collabora- 
tion with several colleagues at Harvard, the Newton School De- 
partment, and the New England Education Data Systems . 1 The 
plan for the construction of the isvd deliberately called for si- 
multaneous revision of the software that controls a real-time 
interactive computer response, with the purpose of making it 
possible for a person and the computer programs with which he 
interacts to understand each other— not fully, only better— in a 
series of interactions. 

The simultaneous development of computer and guidance 
softwares for the isvd has created a cyclical interaction: guidance 
demands cause change in computer software, and change in 
computer software in turn provides capability for new guidance 
functions. The experience has indeed been a powerful one and 
has brought me to understand something of the ways in which 
form and idea modify each other as they evolve. 

Throughout my service on the Commission on Tests, I have 
become aware that testing in the College Entrance Examination 
Board occurs within infrequently examined restraints. One of 
these is that the test theory necessary for purposes of the Board is 
fully developed and hence unmodifiable. Another is that the 
mission of the Board is to illuminate selection of applicants to col- 
lege. Members of the Commission have questioned both of these 
restraints in several ways, as the briefs in this report attest, but 
further deep consideration of them will in the future be required 
of many others before a sharp, effective, and inexpensive re- 
sponse will become possible (if indeed it really is possible). 



i. Principal Investigators of the isvd are: Russell Davis, Richard Durstine, 
Allan Ellis, Wallace Fletcher, Edward Landy, Robert O’Hara (Executive Di- 
rector), and David Tiedeman (Chairman). 
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Attacking the Question 

By seriously addressing the question, “Can a machine admit an 
applicant to continuing education?” I believe it is possible to 
understand and describe a considerable break with the tradition 
of testing, a break that I think the College Board should con- 
sider undertaking. It is my view that the Board has done as well 
as it can under the present theory of testing, which it and Edu- 
cational Testing Service had a considerable hand in both shaping 
and using, and that new theory is necessary if a truly improved 
Board admissions program is to emerge. My purpose is to sketch 
one approach to a new theory. 

I am using the word “machine” in this paper rather than 
“computer,” because although I want to suggest that a new 
theory can emerge from the planning and implementation of a 
sophisticated real-time interactive computer system, I also want 
to deal only with making a procedure, not necessarily with 
making that procedure operate in a computer without other aids 
or human response. Thus, for my purposes, a statement of a 
procedure will also in effect describe the machine needed to 
carry it out. There need be no existing machine capable of carry- 
ing out every stated procedure. Nevertheless, a computer actu- 
ally happens to be one form of a machine that makes concern for 
my question interesting. Computer programs define procedures 
that the computer is to execute and thereby make the computer 
become one machine after another. Such procedures can be di- 
rectly written by applicants themselves as well as by psycholo- 
gists and computer programers. 

It might be suggested that the machine I am proposing could 
actually be an admissions officer, or that it could be proccdural- 
ized to act like an admissions officer. I exclude both these possi- 
bilities because I want a machine that needs only to achieve 
stated ends, not to be various people and thereby bring along 
unnecessary ends and means; and I want one unencumbered by 
needless admissions acts now performed by admissions officers 
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and one able to do different things from those now done by 
them. 

The machine I am proposing should be proceduralized so 
that the ends of the admissions officer and the ends of the ma- 
chine arc identical. It is not necessary that the means of the ad- 
missions officer and of the machine be identical, although there 
is also no reason to avoid making their means alike when doing 
so does not needlessly encumber the machine. 

Purpose of Admission to Continuing Education 

We in the Commission on Tests have never settled on what we 
believe to be the purpose for admission to continuation of edu- 
cation, but my argument here demands that I propose a purpose. 
I therefore suggest that the purpose of admission to continuation 
of education should be the acceptance of candidates likely to 
prove satisfying to the goals of the institution, and the rejection 
of all others. Education should be thought of as continuous for 
each individual, although it may well still have to be provided 
in more than one institution and thus be satisfying to more than 
one set of goals. 

Education should be satisfying, not merely satisfactory. Thus 
I include in the purpose of admission the necessity for the appli- 
cant himself to become satisfied with his education while he is 
being educated. I am implying (i) that admissions officers cannot 
be satisfied merely that their admitted candidates arc satisfactory 
upon admission, and (2) that the symmetry necessarily implied 
in educational satisfaction requires both that the applicant be- 
come satisfied during his admission and during his education 
when he becomes a student and that the institution, as resource 
for enlargement of the student’s intelligence, be satisfied 
throughout the applicant’s admission and later education. Cole- 
man has given the Commission on Tests the concept of sym- 
metry. He elected in his brief (sec page 19) to apply the concept 
only to the evident asymmetry in control of the admissions pro- 
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ccdure. I make reference to the same concept but extend it into 
education as well. 

If continuation of education at any given instant is to prove 
satisfying to the goals of the institution admitting the applicant 
as specified, admitted candidates who by then arc students must 
themselves set goals for the institution. Students who do so give 
the institution chance of viable existence. Students who live by 
personal goals at admission assume obligation during their edu- 
cation to live both by and with the institution’s goals as well. As 
Commission members Fricdenberg, Gordon, Herscy, and 
Schlossbcrg have argued in the Commission, in living with the 
goals of an institution, students become the critics of those goals 
and thereby offer the institution opportunity for its perpetual 
change— and probably its improvement as well. 

Self-Correction and the Satisfying Continuation of Education 

As Coleman proposes that applicants and institutions govern 
themselves in the admissions process, so do I propose that institu- 
tions and the students they arc educating will prove most satisfy- 
ing to institutional goals if both arc subject to expectations and 
to procedures for self-correction . 2 The basic personal process of 
self-correction is creation: something comes into a person’s being 
that was not earlier articulable although it may well have al- 
ready existed tacitly for quite a while. Ideational creativity, the 
goal of education, requires a student to relate himself to his ex- 
perience and his environment so that he is tentative about some 
things and from time to time commits himself to other things. 
These vacillating dual conditions of tcntativcncss and commit- 
ment pennit the mind to play with ideas both as wholes and as 
parts. The wholeness of an idea allows one to deal in theory with 



2. Gannaway, Myra T., Changing Perspectives ht Education and Self-Correcting 
Thinking. Cambridge, Mass.: Harvard Graduate School of Education, 1968. 
Unpublished qualifying paper. 
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its conceptually divided parts. In dealing with its parts one can 
experience ideational aspects of a totality in intensity necessary 
for comprehension of the totality. Frequently the whole sug- 
gests parts not yet envisioned. Occasionally parts coalesce into 
wholes as yet unimagined. However, self-correction is itself ex- 
perienced as a whole, although its emergence can be encouraged 
by dealing with some of its contributing parts. 

Maintaining a sense of self-correction makes demands on 
applicant and institution alike. For the institution’s part, (i) a 
significant number of its officers must be capable of self-correc- 
tion, and (2) the organization of the institution must not contra- 
dict the expectation for self-correcting activity on the part of 
such enlightened officers. Coleman, Cross, Friedenbcrg, and 
Hersey arc among Commission members who have advocated 
this institutional condition in continuing education. For his part, 
the applicant, as Fricdenberg argues (sec page 33), (i) must be 
given opportunity to share in goal determination of the institu- 
tion even at the time of his admission, (2) must perceive this 
opportunity as fairly offered and fairly administered, and (3) 
must be capable of self-correction when he is admitted and 
throughout his education in the institution. 

Science and Self-Correction. Self-correction is an attitude fun- 
damental to science, including of course psychometrics, as well 
as to personal development. I can further elaborate my meaning 
of self-correction in personal development through considera- 
tion of the more commonly understood principles of science. 1 
am referring to the process of science, not necessarily to the 
products of scientists. Comprehension by the scientist is the end 
product of the process of science, and this comprehension ordi- 
narily implies successive differentiation of the concepts involved. 
Personal development requires comprehension of the processes 
of science as well. 

All of us engage in informal testing intermittently in daily 
living whether we are scientists or not. Through such informal 
testing we sharpen in self-correcting ways our understanding of 
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the relationship between ourselves and our experience and en- 
vironment. Scientists formalize such thought, or tests in which 
they arc willing to think one way or another depending on the 
outcome of an observation contingent upon a prior supposition, 
and call it “doing science . 11 Scientists “do science” with con- 
cepts they have formed about the relationships among them- 
selves, their concept, and the experience they are attempting to 
understand by means of their concept. This idea about science 
is not novel. Polanyi relics on it . 3 Bronowski explicitly grounds 
the identity of man in it , 4 as I am suggesting be done here. 

The primary paradigm I am proposing for self-correction in 
personal development by “doing science 11 is that every man 
seeks clarification of the relationship between an idea he has and 
his experience (including others 1 summaries of that experience). 
He thus brings his ideas into focal awareness in the ground of his 
and others 1 experience. If power is expected from this primary 
relationship he must study it in a secondary relationship in which 
he secs himself both as the inventor and evaluator of the idea and 
as the agent of his experience. 

Actually it is impossible to separate these primary and sec- 
ondary relationships; one continually interacts with the other. It 
is therefore necessary to conceive the two as figure and ground 
in phenomenological interaction. What ordinarily eventually 
happens when comprehension takes place is that concept and ex- 
perience are given primary position as figure, within the ground 
of the “I-and-concept 11 relationship. The processes by which 
ideas are related to experience occur normally. However, it is 
possible for individuals to become much more aware of them, 
and for that awareness to sharpen decision-making activity, par- 



3. Polanyi, Michael, The Tadt Dimension. Garden City, N.Y.: Doubleday and 
Co., Inc., 1966. 

4. Uronowski, Jacob, The Identity of Man. Garden City, N.Y.: The Natural 
History Press, 1965. 



ticularly personal responsibility for decisions. As Carroll has 
suggested to the Commission on Tests, comprehension of the 
process itself matures very slowly in man’s cognitive develop- 
ment. It may well occur at different rates at different times in 
different people’s lives as Carroll, Gordon, Holtzman, Pace, and 
Schlossbcrg have contended in the Commission, I mean to say 
only that the effect can be helped to occur sporadically through- 
out life. Commission members Gordon, Hersey. Hoy, and 
Schlossbcrg seem to hold a similar belief. Comprehension of the 
general clement of the process occurs even more slowly, but 
there is a good deal of evidence that it is indeed a capacity man 
can develop. 

Formal testing differs importantly from informal testing in 
that formal testing must be conceived in public terms and in- 
formal testing can occur largely in private terms. 

“ Doing Science ' and Formal Testing. The scientist tries to 
bring into the public realm the understanding he achieves as a 
result of informal tests of his concepts and experience. As he 
makes this move from the private to the public realm, he finds 
himself explaining not only what he knows but also the bases 
upon which he claims to know it. These bases, as they enter the 
public realm, become the material others can use to examine the 
scientist’s impression of what he understands. In the pursuit of 
science, the bases occasionally become formalized and serve as 
tests that other people in turn apply to the relationship between 
concept and experience that the scientist claims he understands. 
As this process occurs, investigations that were formerly “fluid” 
in Schwab’s language 5 or “whole” in my language of self-cor- 
rection become “static” in Schwab’s language or “partial” in 
my language of self-correction. 

In static or partial investigations, the bases of understanding 
are kept fixed while the realms of application of those bases are 



5. Schwab, Joseph, The Teaching of Science as Enquiry. Cambridge, Mass.: Har- 
vard University Press, 1962. 






David V. Ticdeman 



varied. Static investigations therefore ordinarily expand and 
clarify just the limits of application of an original understanding. 
Fluid or whole ideation, on the other hand, is ordinarily rela- 
tively free of the restraints placed on static investigations. Such 
freeing in turn allows concepts and experiences to be “seen” and 
presented in new lights. When the freeing is a superordination of 
previous restraints, science or persons arc said to progress. When 
the freeing is the establishment of new restraints but in a different 
field of awareness, science or persons arc said to become diversi- 
fied. 

Means and Ends. My argument so far first noted that the pur- 
pose of admission to continuing education should be the accept- 
ance of candidates likely to prove satisfying to the goals of the 
institution and the rejection of all others. I then argued that the 
sense of self-correction must be preserved if students arc to 
achieve a continuation of education satisfying to applicant and 
institution alike. I finally noted that awareness of doing science 
in personal living, particularly the doing of formal testing in 
science, is the aspect of self-correction critical to a satisfying 
education. 

I next want to point out the existence of a fundamental flaw 
between the way tests arc now being used in admission to con- 
tinuing education and the end of cultivating self-correcting ac- 
tivity. The flaw is that applicants arc not collaboratively in- 
volved by admissions officers in the problem of specifying goals 
and pursuing them. Tests by themselves offer no present oppor- 
tunity to correct that situation, and it is this flaw that the ad- 
missions machine specified in this brief would seek to eliminate. 

An Admissions Machine 

If the sense of self-correction I have defined here is not to be seri- 
ously contradicted during admission to continuing education, 
that admission should be carried out as an integral part of the 
emerging self- determined and self-corrected career, a particular 
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aspect of self-development. The cultivation of the awareness of 
doing science, particularly the awareness of doing formal testing, 
in living can be achieved during admission if the needed ad- 
missions machine is planned as a part of a career machine. One 
career machine is now under specification and partial construc- 
tion as an Information System for Vocational Decisions (isvd). 

Role of the Career Machine. In an isvd, a relationship is ar- 
ranged between an inquirer and certain facts. In my particular 
isvd, facts arc of four kinds: (i) occupation; (2) military service; 
(3) education; and (4) personal and family living. The isvd ex- 
pects the inquirer to turn facts and data into information. The 
turning of facts and data into information is cultivated by four 
machine functions. Each of the first three— exploration, clarifica- 
tion, and planning— is in the anticipation phase of the Tiedeman 
and O'Hara paradigm of decision-making development. 6 The 
fourth function, review, is in die accommodation phase of that 
paradigm. The expectation that facts and data will be turned 
into information is in turn supervised or monitored by the ma- 
chine. The monitoring is governed by the assumption that the 
person will become self-correcting in his use of these functions. 
The principal goal of that self-correction is the mastery of the 
paradigm of decision-making development. The comprehension 
of this epigenetic phenomenon of decision-making development 
is the end product expected of repetitive use of an isvd. The 
comprehension probably occurs develop mentally in ways as yet 
unknown, but it undoubtedly occurs over an extended period, 
probably the period of the life itself. 

The major condition potentially leading to the comprehen- 
sion of decision-making development is the monitoring function 
just mentioned. The monitoring function is programed to detect 
and report occurrences in which the inquirer is seemingly pro- 



6. Tiedeman, David V., and O'Hara, Robert P., Carm Development: Choice 
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jecting extraneous information at times when the system ex- 
pects him to assume responsibility for his activity and under- 
standing during exploration, clarification, planning, and review 
about the current, past, and likely future points of decision in his 
career development. The inquirer will be able to ask for the 
basis of the system’s judgments at any time. The system will be 
programed to report those bases and to invite the inquirer to 
modify them if he is so inclined. The modification can be ac- 
complished by having the inquirer program the computer him- 
self for solutions to problems he defines. This procedure will 
make the isvd “reveal itself,” or at least its procedures, a condi- 
tion Ellis and I declare to be the major object in counseling . 7 
Should the inquirer act on the machine’s invitation to modify 
the bases for system monitoring of his own interaction, he begins 
the construction of a personal or truly esoteric information sys- 
tem. The degree to which the inquirer can either extend or 
change the public monitoring function represents the degree to 
which he can plumb his intuitions and make them articulate 
using the ways of expression delineated by the system. The in- 
quirer is thereby helped to make his tacit understandings expli- 
cit, at least to himself, and probably to others should he elect to 
do so. 

The assumption and practice of responsibility for monitoring 
one’s activity while inquiring cause understanding of a particular 
decision-making development. The comprehension of the way 
in which one is interacting with the system to get such an un- 
derstanding is the epigenetic process of decision-making devel- 
opment. When this experience occurs frequently, the inquirer 
becomes explicitly its master, and in reality thereby becomes 
master of his self. 

The isvd system will use the Tiedeman and O’Hara para- 
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digm of decision-making in vocational development as an ex- 
plicit model that inquirers will be expected to master through 
repeated use of the isvd. More important, through interaction 
with the system and with counselors who are aware of and at- 
tempting to facilitate the more general effect, inquirers are ex- 
pected to master the epigenetic process of decision-making de- 
velopment. Inquirers having such mastery are skilled in the use 
of purposeful action , 8 of self-correction , 9 and of “doing science” 
or reasoning either in the sciences themselves or in the humani- 
ties as well. 

The Admissions Machine in Broad Outline. The existence of 
an isvd career machine would simplify the admissions process if 
applicant and admissions officer both believed that the other was 
self-correcting and, as Coleman advocates in his brief (see page 
19), mutually decided to share facts and data in order to collabor- 
ate in deciding whether this particular applicant should be ad- 
mitted to this particular institution or not. The prior uses that the 
inquirer had made of an isvd would give him a means of now 
characterizing his decision to apply to a particular institution so 
that the admissions officer could, with the inquirer’s permission, 
be privy to how he had made his decisions, not just to the fact 
that he now wanted to be admitted. The admissions officer could 
examine this record for detail, complexity, integrity, and self- 
correcting activity. He could also plumb the record for the goals 
the applicant wanted to fulfill and for the applicant’s justification 
that such goals could be fulfilled collaboratively at the particular 
institution with its present stated goals and existing procedures 
for self-correction of these goals. Final evaluation would remain 
in the hands of the admissions officer, as it must be. 

The self-correcting processes that Bronowski 10 and Ganna- 
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way 11 define depend on the individual’s capacity to examine 
honestly and continually the relationship of himself to his experi- 
ence. When an individual is doing so he is in reality acting as a 
scientist about his self-processes. I believe that the major issues 
in such an examination at the time of application to continuing 
education arc (i) the nature of the outside as that outside is 
known to others, and (2) the nature of purpose 12 as its nature can 
be known collaboratively to applicant and admissions officer 
alike. In order to facilitate and demonstrate these conditions the 
admissions machine should contain an admissions game that can 
be used by an applicant first in familiarizing himself with an- 
other’s experience about admission to continuing education and 
then in showing a particular officer how he used that game in 
his personal decision. The admissions game would constitute a 
giant step in helping applicants maintain individual initiative 
within a sense of understanding of institutional complexity, a 
condition Coleman has urged the Commission on Tests to try to 
bring about. 

A further widening and deepening of the context for collab- 
orative testing action between applicant and admissions officer 
can help both to move the admissions decision from its present 
base, which is almost gamelike, to a more intimate base in which 
the aspirations and evaluations of applicant and admissions officer 
arc more pcnctratingly known to both. The instrument for such 
widening and deepening can be a computci^based interactive 
admissions system, or an admissions machine. In such systems, an 
admissions officer can combine both the admissions game and his 
subsidiary decisions. The embedding of game and decision bases 
into an interactive information exchange, which permits some 
applicant programing, would naturally expand the area of appli- 
cation of the admissions officer’s thought about applicants and 
their intentions and accomplishments. The use of such an exten- 
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sive system would give applicants a sense of participation in ad- 
mission to their continuing education, which standardized test- 
ing docs not now make possible. 
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Implications of an Admissions Machine ; 

for Test Theory and Practice j 

Although the admissions officer would gain from the availability 
of an isvo-likc self-correcting record of an applicant’s career, 
there will undoubtedly be additional information he would like 
to have, certain kinds of factual information as well as test in- 
fonnation. I shall not say more about needed factual information 
because it is likely to be specific to an institution and should be j 

planned and obtained with that expectation in mind. However, 
the problem of testing itself is the principal interest of the Com- 
mission on Tests, and it can be treated much more thoroughly in 
relation to a potential admissions machine. 

The Process of Science and Admissions Testing. The process of 
science contributes in two ways to admissions testing. First it 
forms a means of showing what admissions testing now is. Sec- 
ond it suggests a model of what admissions testing might become. 

Admissions testing now takes place under static conditions. 

For all practical purposes, psychometricians conceive admissions 
testing merely as an instrument an institution can use to find the 
applicants it wants by comparing them with classes of the years 
past. As Thresher has indicated in his brief for this report (page 
146), the present procedure of formal admissions testing thus 
constitutes only a feedback loop— that is, the criterion is fixed and 
the test and associated admissions studies provide probability data 
in relation to the fixed criterion. A feedback system is static as far 
as its restraints are concerned. One of the ways in which a feed- 
back system has been moved toward a dynamic or 1110a' fluid 
condition, in which feed forward then starts to be available, has 
been to use the results of feedback obtained within existing con- 
straints to make the feedback operate to correct the direction in 
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which an object is moving. This is the sense in which feedback 
operates in missile guidance systems, in power steering of auto- 
mobiles, and the like. This is also the sense in which psychomct- 
ricians now construe guidance as based in existing test theory. 

If the planning function of my isvD-like career machine were 
constructed and used as an admissions game in an admissions 
machine, the Board would create a first movement from a static 
test system to a dynamic information-creating system. Although 
the static system of college admissions based in present Board 
tests is for the most part deterministic, the effects of some of that 
determinism could be somewhat alleviated in the students sub- 
ject to it by causing them to comprehend the “theory of the ad- 
missions game,” as well as to take tests processed in secret by 
admissions officers. This is equivalent to the takeover of the pub- 
lic monitor by the inquirer in the isvd career machine. Students 
who become expert in such a game are more likely to petition 
for admission to continuing education on grounds that allow 
them to act a little more intuitively within both the restraints of 
the present “game” and the operation of those restraints on their 
desires. 

Widening the Data Context in Formal Testing . It is now possi- 
ble to widen the context of data processed by admissions officers 
without sacrifice of either accuracy or accountability. One form 
of widening would be to incorporate the patterns resulting from 
modularly arranged programed instruction, as Commission 
members Coleman, Holtzman, and Suppes have suggested. A 
second form of widening can be accomplished by shifting the 
basis of existing Board questions from multiple response to free 
response as Thresher suggests in his brief (see page 146). This 
alternative has not in the past been fully exploited because it has 
been said that it takes too long to process answers in the short 
turn-around time between receipt of answers and need for pro- 
cessed results and that it is impossible to get readers to agree on 
evaluations of responses. 

As to the feasibility of processing frec-rcsponsc tests cffici- 




174 



cntly, the availability of computers now makes it possible to 
pocess in short order a response entered into the computer by 
an applicant. In fact, the computer can convey its response to the 
applicant himself as well as to any other party permitted access 
to the question-response sequence. Thus the matter of time be- 
comes irrelevant if testing is programed so that questioning, 
answering, and processing arc done almost simultaneously. This 
effect is already available through systems software permitting 
students to learn by problem-forming. 

As to the second objection, regarding the reliability ofcvalu- 
ating such frcc-rcsponsc questions; Stone and his associates have 
developed a procedure for computer recognition and response 
to key words in context . 13 Fred Ferris picked up this processing 
idea and exploited it in relation to providing items on Board 
tests that can be presented as problems, not as a given set of pos- 
sibilities from which a selection is made . 14 Probably there arc 
others as well, but I do not know of their work at this time. This 
procedure markedly changes the nature of test con tern as Ferris 
discovered in his exploratory work. It isn’t possible just to take 
old questions and expect answers in new forms; the forms of the 
questions must be changed as well. 

If the processing of key words in context is done wliilc the 
applicant is interacting with computer programs, further gains 
become possible. These further gains as already achieved by 
Commission member Suppes in his concept-forming instruction 
arc: (1) the responses identified by the computer can be relayed 
to the applicant before he leaves the system, and (2) the appli- 
cant can be asked to verify or revise such identifications before 
he leaves the system. The applicant’s revisions can in turn be 
made p matter of record and report to the admissions officer. 



13. Stone, Philip J.; Dunphy, Dexter C.; Smith, Marshall S.; Ogilvic, Daniel 
M m The General Inquirer: A Computer Approach to Coutent Analysis, Cambridge, 
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14. Personal communication. 
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This procedure therefore bypasses one of the major problems of 
reliability, namely keeping the process sensitive to what the ap- 
plicant intended. An additional gain as preliminarily modeled 
by Ferris is that the computer program can also report to the 
applicant the test-maker’s judgment of the complexity and com- 
pleteness or accuracy of his answer. This is the new form of re- 
porting that Thresher advocates in his brief (page 146), not just a 
reporting that merely gives the correctness or incorrectness of a 
response. I am thinking of a response processed according to the 
numerous alternatives for tackling a problem. The test-maker 
provides scoring of the response in outside terms and then uses 
that outside scheme both to score the response in his terms and to 
share the report of such scoring with the applicant. If the appli- 
cant notices anything awry with his score, he can report such 
impressions. This eliminates another issue in reliability. 

The major reliability issue in test theory has to do with (1) 
the limited number of questions— the sampling— included in any 
one tost, and (2) the assessment of the possibility that the level 
of a person’s score on a sample of test rems will remain relatively 
constant in relation to the scores of comparison groups tested 
with different items. Limiting subject-matter coverage in a given 
test by sampling is indefensible. It may prove valid with a few 
things like sets of arithmetical operations, but in a real sense each 
question is a unique question. When different questions arc 
placed in sets, their categorization can be challenged by anyone 
who knows the subject. In test-construction committees at the 
Board, placement of questions in categories is ordinarily done 
conscnsually, after discussion, not individually. However, the 
placement of questions into tests is a serious problem that proba- 
bly shouldn’t be delegated by admissions officers to test commit- 
tees. Are there ways in which admissions officers can deal with 
such decisions t’» mi selves? There prove to be ways when the 
selection of questions for answers is left to the applicant and 
when the entire set of questions from which the applicant can 
draw is known to the admissions officer. If an applicant is allowed 






to respond to questions in computer programs and if the process- 
ing of responses is arranged as I have noted above, then the rec- 
ord of response transmitted to the admissions officers can be not 
only the applicant’s processed answer to a problem (complexity, 
completeness, accuracy) but also general statements (in the par- 
ticular admissions officer’s own terms, should he desire such a re- 
port) of what categories of questions were attempted and what 
categories of questions were not attempted. 

But how then could an admissions officer compare one appli- 
cant with another? Coleman contends that this is the major ques- 
tion for an admissions officer faced with selecting a limited num- 
ber of applicants, when he reviews a set of applicants’ test 
responses. The psychomctrician’s habit of doing a lot of the de- 
ciding for an admissions officer is no longer necessary. It is now 
possible to put before admissions officers themselves both data 
on the applicant’s choice processes, with histories of the admis- 
sions officer’s decisions about prior candidates, and much of the 
data on which the Board’s test committees act themselves. It is 
additionally possible to get information to an admissions officer 
before he is required to act on it, in a form in which it has been 
previously dealt with by each of the applicants to his institution. 
Hence he would have an opportunity to form his impressions of 
each applicant both by reviewing information about him indi- 
vidually and by comparing him with other applicants. Further- 
more, for admissions officers who want such help, it would be 
possible to work out computer programs partially summarizing 
whatever information an admissions officer might designate that 
is available in the content of a test. Doing so would preserve the 
Board’s traditional value of providing valid and condensed in- 
formation for quick action. It might also provide the public 
means both to monitor decision-making processes of admissions 
officers and to accumulate data for C. Robert Pace’s College & 
University Environment Scales and James Coleman’s Admission 
Game. 

Determining Goals in Self-Correcting Institutions . Institutions 
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pursue their own objectives on the premise that their use of stu- 
dents who are their human resources actually benefits society. 
Their goals and the judgments on which they are advanced, 
however, are currently subject to serious questioning by stu- 
dents. I don’t mean to favor one person’s goals over another’s. 
But I do argue the applicant’s right to make an institution aware 
of his intended use of it, as well as the reverse. The young indi- 
vidual must come to know that his society is in him as well as he 
in it. As Coleman indicates in his brief (page 19), this sense of 
being an individual is not readily come by in large modem or- 
ganizations. At the present time young people arc kept in edu- 
cational navigational systems for a long time and then released 
to personal navigational systems with little or no effort to culti- 
vate what this shift entails. I know that this assertion is subject 
to debate. Nevertheless I make it in company with Commis- 
sion members Gordon and Schlossberg on the grounds both of 
the limitations of modern educational guidance and of the in- 
adequacies of the theory on which it is based. 

If the Board is to improve the theory on which guidance 
operates, it will have to become involved in the basic substance 
of guidance, namely the self. Gordon, Schlossberg, and I do not 
use the concept “self” loosely. We are aware of its history in 
metaphysics, and we too find it mercurial. For discussion here I 
am using Bronowski’s definition, 15 and therefore I am talking 
about the grounds that are available to personal awareness. I 
merely extend Bronowski’s belief into my own tha*. compre- 
hension of the processes by which that availability occurs can be 
facilitated. 

I believe that self-awareness deepens and widens with the 
comprehension of the processes of choosing. Admission to con- 
tinuing education is one context for choice. In this context, the 
applicant is required to place what he knows injuxtaposition with 
what he wants. He is then in a position to advance what he 



wants in terms of what he knows, using the resources of an insti- 
tution of continuing education as a means for his intention and 
plan. If the admissions officer is to have sufficient inkling of an 
applicant’s intention and plan, he must have a context in which 
what the applicant knows can be advanced to him in relation to 
what the applicant wants. This pair of conditions becomes pos- 
sible in a computer-based interactive exchange system such as 
the career machine. It would be better grounded if it originated 
from free responses to problems, as I have advocated for the ad- 
missions machine, but free response is still not enough. What the 
admissions officer should additionally seek is information con- 
cerning the applicant’s ability to form problems, not just to solve 
them. If an applicant can only solve problems, he tends to know 
and interact with the world largely in another’s terms, not his 
own, but if he is able to form problems, he is able to react to the 
world in his own terms as well as in another’s. If the admissions 
officer watches the extent to which an applicant includes the 
views and purposes of others and advances his own purposes, he 
can learn a lot about how the applicant accomplishes goals and 
uses information. 

Conclusions 

I posed my question “Can a machine admit an applicant to con- 
tinuing education?” without expecting that it could be answered 
affirmatively. Instead I merely proposed that it be taken seriously 
in order to gain new perspective on the theory and practice of 
testing. In attempting to create this perspective I defined my 
purpose of an institution of higher education in its admission 
process as the identification of applicants likely to prove satisfy- 
ing to the goals of the institution. I then proposed that that pur- 
pose be expanded to incorporate the goal of self-correction and 
the procedures of science as means compatible with achievement 
of that goal. 

Having proposed a new purpose for admission to the con- 
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tinuation of education, I outlined an admissions machine con- 
sistent with its attainment. I finally used the admissions machine 
to adumbrate the assumptions of formal testing in order to pin- 
point assumptions in the theory of testing that can be changed as 
the practice of admissions embraces the concept of an admissions 
machine. 

What I propose as necessary will be difficult to accomplish. 
As do Commission members Friedenberg, Gordon, Hersey, 
Hoy* and Schlossberg, I essentially suggest that the major pur- 
pose of education is to help persons clarify their own relationship 
to language and experience. Admission to continuing education 
must be consistent with that purpose and thus should be offered 
in expectation of self-correcting activity on the part of an ap- 
plicant and in an atmosphere in which the applicant will agree 
that such has fairly been the case. 

My proposals challenge cherished assumptions in testing and 
in admission to continuing education, two activities that I have 
come to know as relatively inseparable at the present time. Test- 
ing theory is largely defined in terms of admissions purposes; 
admissions practices largely follow test theory. The concept of a 
machine has been my principal means of being explicit and of 
proposing how my ideal is within realization. Not only do I 
make no claim that what I propose will be easy to attain; I do 
not even make a claim that a machine can admit an applicant to 
continuing education. However, I do claim that examination of 
the question as if it had an affirmative answer has offered a new 
way to question fundamental purposes and means in the college 
admissions process. To fail either to question in the terms here 
advanced or to act on new premises will be to abandon present 
institutions of higher education to new institutions in which the 
self-correcting activity now being sought by college students 
will find more friendly haven. 
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Pearson’s brief (page 100) outlines a way in which the Commis- 
sion on Tests recommends that the College Board bring com- 
puter technology into programs that for the most part already 
exist. His proposal is based on assessment of the state of com- 
puter technology and on full knowledge of the way in which 
the Board ordinarily changes its approach. Unfortunately there 
is no experience to assess in the construction and implementation 
of an admissions machine, but there are numerous parts that 
could be assembled and experimented with before attempting 
to engineer the whole machine: (i) materials in a number of the 
current Board financial, guidance, and test programs; (2) niodu- 
larly arranged programed instructional material such as Com- 
mission member Holtzman advocates and Commission member 
Suppcs has already produced; (3) laboratory instruction via com- 
puter as was outlined in Holtzman’s computer conference; (4) 
the test-item files at ets and reported experimentation with in- 
dividually tailored tests; (5) concept-formation teaching pro- 
grams such as Suppes has constructed, others have elaborated, 
and several people adumbrated at Holtzman’s computer con- 
ference; (6) the valuing procedure Martin Katz is constructing 
at ets for computer aid in bringing values into focal attention 
during educational and vocational deciding; ( 7 ) the isvd; and 
(8) Pearson’s advice about bringing educational technology into 
Board programs. 

However, merely assembling these parts will not create a 
feasible, efficient, and economic system that will markedly 
change the character of, procedures in, and probable satisfaction 
with admission to continuing education. These results will only 
come with intimate, prolonged, and careful systems design and 
construction by admissions and computer experts working col- 
laboratively. 

It is recommended that the College Entrance Examination Board : 
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i. Establish a small study group that will outline an admis- 
sions machine and undertake feasibility studies needed to reach 
within two years: 

a. a better grounded decision on the further possibility of de- 
veloping an admissions machine; 

b. if then still considered possible, a more detailed financial 
and structural plan for implementing what appear to be better 
lines of its development. 

(2) In appointing this study group, recognize that its report 
within two years will bear serious financial implications for the 
Board. 



Record of Endorsement, Comment, Reservation, 
and Dissent on the Recommendations in 
Can a Machine Admit an Applicant 
to Continuing Education? 

It is recommended that the College Entrance Examination Board: 

1. Establish a small study group that will •. * nc an admissions 
machine and undertake feasibility studies needed to reach within 
two years: 

a. a better-grounded decision on the further possibility of 
developing an admissions machine; 

b. if then still considered possible, a more detailed financial 
and structural plan for implementing what appear to be better 
lines of its development. 

Endorsed by 16 members: Blocker, Carroll, Coleman, Foster, 
Friedenbcrg, Gordon, Hcchingcr, Hcrscy, Holtzman, Hoy, 
Marland, Pace, Pearson, Schlossbcrg, Thresher, Ticdeman. 

Carroll: “My only reservation is that the proposed ‘admis- 
sions machine’ may actually be partially redundant with some 
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other proposals. Also, I do not like the implications of calling it 
an ‘admissions machine/ Some other phrase would be more 
graceful.” 

Foster: “Depending on the cost. But I doubt if it makes sense 
until a career-college machine is further perfected— either one 
like Tiedeman’s isvd at Harvard or Katz's sigi at ets. Perhaps 
this is Tiedeman’s suggestion.” 

Gordon: “I have some concern about the name ‘admissions 
machine.' I think Tiedeman is talking about a ‘continuing edu- 
cation decision assistance machine'— -a long title but more accu- 
rate than ‘admissions machine.' The machine does not admit.” 

Not endorsed by 2 members: Prentice, Shirley. 

1 member abstained: Abelson. 

2. In appointing this study group, recognize that its report within 
two years will bear serious financial implications for the Board. 

Endorsed by 13 members: Blocker, Carroll, Coleman, Gordon, 
Hersey, Holtzman. Hoy, Marland, Pace, Pearson, Schlossberg, 
Thresher, Tiedeman. 

Coleman: “This should be combined with Recommendation 
1." 

Not endorsed by 4 members: Foster, Friedenbcrg, Hechinger, 
Shirley. 

Foster: “Why? Not necessarily.” 

Hechinger : “In not endorsing this recommendation, I merely 
want to note that the research indicated does not imply a com- 
mitment to develop the machine unless the findings are com- 
pelling.'' 

2 members abstained : Abelson, Prentice. 
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General Comments 

Coleman: “I believe the principal emphasis of any interactive 
procedures designed to aid the matching between college and 
applicant should be toward providing the kind of possibilities 
for applicants described in Hcrscy’s brief. I believe the proce- 
dures recommended for study in the present brief do tend in this 
direction, and as a consequence I favor such a study. But I be- 
lieve the implementation of this brief’s recommendations and 
the Hcrscy recommendations would necessarily be joint, since 
the overlap is great.” 

Foster: “A most stimulating idea. Applying these principles 
rather than actually building the machine mighl elicit most of 
the new ideas for admissions procedure. One wonders if, given 
the concepts of improved approach provided here, the amount 
of money (should it be available) necessary to carry Tiedeman’s 
suggestions into operation might be equally effective if used to 
improve person-to-person guidance. I cannot accept the impos- 
sibility or diseconomy of training people to help perform these 
functions in a way satisfying to the student.” 

Holtzman: “I have mixed feelings about this, but, on bal- 
ance, I think some statement having to do with computer-as- 
sisted guidance and college admission is a good idea to pursue 
further with a study group.” 

Marland: “While I support this brief in its conceptual context 
as feasibility study only, 1 have very serious reservations about 
the ultimate procedure.” 

Prentice : “Unless and until the research and conceptualiza- 



tion underlying isvd develop beyond their present stage, I , 

would consider this proposal as risking a long expensive journey j 

down a blind alley. It is a fit subject for Harvard’s Graduate / 

School of Education, not for the Board at present.” i 

Shirley: “I do not endorse these recommendations primarily 
because they are using the College Board as a sponsor of research I 

rather thr.n realizing that its first responsibility is operational. j 
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The costs anticipated are unrealistic and would absorb the full 
budget and staff of the Board. The ultimate development would 
be beyond the capacity of either the students or the Board mem- 
bership to absorb. 

“Refer to my comments on the recommendations of both 
Hersey and Pearson. In all these cases of symmetry, I am sym- 
pathetic to the idealism involved but realistic enough to know 
that the Board docs not have and probably cannot get the sup- 
port that will make possible the translation from dream to real- 
ity. As this is the case, the Board should be a facilitator of the 
translation process; it should not be asked to subsume it. 

“Our recommendation to the Board should be that it recog- 
nize the need for this kind of symmetry and use its influence and 
good offices to (a) get financial assistance— from a foundation or 
government— to do the research and to make the background 
studies needed to develop the plans for the major program and 
(b) bring the full force of the association membership to l ir on 
government (the only agency that could fund such a venture) to 
give it full support. But the Board must ^ t accept responsibility 
to do what is patently impossible at the outset. 

“I must add that I’m intensely skeptical of the computer ap- 
proach. The Harvard demonstration showed that the first steps 
are tremendously costly and exceedingly halting, even under 
ideal conditions. Considering the magnitude of the problems of 
gathering data of the kind needed from 2,000 or more autono- 
mous colleges and universities and from X million high school 
students and their families, updating these data to keep the in- 
formation current and useful, programing everything to make 
it accessible to every student, even assuring access to compatible 
computers to all these students — this makes the task of training 
more than 100,000 excellent counselors, fully informed of all 
education requirements and developments in the nation and the 
full psychological needs of individual adolescents, seem to be a 
simpler and easier solution to a difficult problem. Have we con- 
sidered all alternatives to symmetry? I fear we have not.” 
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Thresher: ‘i urge that the project be moved into the stage of 
reasonably large-scale involvement with actual students as soon 
as possible. At present it has more anatomy than physiology, 
more bones than flesh.” 
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Recommendations Proposed 
in Commission Members’ 
Briefs 



It is recommended that the College Entrance Examination Board: 

1 . 1 Propose to an organization such as the American Council 
on Education that it create a Council on College Admissions 
and Scholarship Testing that would invite the various testing 
organizations and agencies to join for the purpose of cooperating 
in research that would lead to mutual agreements among them- 
selves and the highcr-cducation institutions concerned, whereby 
colleges would accept scores on different tests as equivalent for 
admissions purposes. 

i .2 Propose that the research and development to be sponsored 
by this council be based on a method of establishing equiva- 
lencies in terms of equivalent predictions of criterion variables. 

(Proposed by John B. Carroll in “Redundant Testing”) 



2.1 Support research to investigate the feasibility of recasting the 
sat along the following lines: 

a. Provision of a section or sections more clearly emphasizing 
developed verbal skills relating to language comprehension and 
learning from language, with the reasoning component dc- 
emphasized; 

b. Provision of a section or sections more clearly emphasizing 
ability in reasoning and inference that docs not depend greatly 
on verbal skills; 

c. De-emphasis of the quantitative sections of the tests, with 
provision for testing of developed ability in this area in specific 
achievement tests apart from the sat; 

d-. Administration of some parts of the sat in an auditory 
mode, to reduce dependence of those parts on reading ability; 

c. Provision of a section or sections that would measure scho- 
lastic learning capacities or potentials as much as possible apart 
from developed abilities; 

f. Validation of sat materials not only against overall abso- 
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lute success in higher education (in various types of programs), 
but also against gains made in developed abilities. 

2.2 Introduce on a gradual basis, if the above types of tests prove 
feasible and acceptable, a new and possibly renamed sat, with 
appropriate dissemination of information about the test both to 
educational institutions and to candidate populations. 

(Proposed by John B. Carroll in “Possible Directions in 
Which College Entrance Examination Board Tests of Abilities 
and Learning Capacities Might Be Developed”) 

3. 1 Systematically gather and publish relevant information about 
colleges, the nature of that information to be determined by a 
working group including representatives of colleges, students, 
and high schools. 

3.2 Beyond the standardized testing that allows colleges to make 
direct comparisons of a few characteristics of applicants, provide 
the applicant with the opportunity, through selection from a 
very wide variety of special tests ranging in subject from numis- 
matics to algebraic topology, to exhibit his special capabilities, 
developed in or out of high school, that may be relevant to the 
colleges’ interest in him. 

3.3 Provide colleges and applicants with information about the 
other only if they are willing to allow information about them- 
selves to be made available. 

3.4 Modify its governing structure to embody a symmetry of 
college and applicant interests. 

(Proposed by Jam~s S. Coleman in “The Principle of Symmetry 
in College Choice”) 

4.1 Offer not merely achievement and aptitude tests but also 
diagnostic tests that would permit colleges and students to select 
each other with due regard for prospective compatability. 
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4.2 Offer high school juniors and seniors, or college students in- 
tent on transfer or graduate study, the opportunity to take tests 
for the purpose of obtaining scores, interpretable by reference to 
national norms in lieu of grades and, if legally possible, in lieu of 
school attendance. 

(Proposed by Edgar Z. Friedenberg in “Can Testing Contribute 
to the Quest for Community Among Students?’’) 

5.1 Explore possibilities for adding to its quantitative reports on 
the performance of students, reports descriptive of the patterns 
of achievement and function derived from the qualitative analy- 
sis of existing tests. 

5.2 Explore the development of test items and procedures that 
lend themselves to descriptive and qualitative analyses of cogni- 
tive and affective adaptive functions, in addition to wider spe- 
cific achievements, 

5.3 Explore the development of report procedures that convey 
the qualitative richness of these new tests and procedures to stu- 
dents and institutions in ways that encourage individualized pre- 
scriptive educational planning. 

5.4 Explore the development of research that will add to under- 
standing of the ways in which more traditional patterns of in- 
struction will need to be modified to make appropriate use of 
wider ranges and varieties of human talent and adaptation in 
continuing education. 

(Proposed by Edmund W. Gordon in “Toward a Qualitative 
Approach to Assessment”) 

6.1 Carry the principle of symmetry to its full realization by es- 
tablishing as soon as possible a College Entrance Service, the 
purpose of which would be: 

a. To guide applying students to colleges that would fit their 
needs and abilities; 
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b. To assist institutions in building student populations that, 
along with their faculties, would enable them to become com- 
munities manifesting their own best possibilities; 

c. To supply diagnostic and guidance materials on themselves 
to individual students; 

d. To supply evaluative materials on themselves to colleges. 

6.2 To this end to undertake simultaneously both operations and 
research, using both printed and computerized techniques. 

(Proposed by John Hersey in “A College Entrance Service**) 

7.1 Increase minority-group representation on the College 
Board stalF to a level that Inore fully reflects the society at large 
and the commitments already shown by member institutions. 

7.2 Change the composition of committee membership to reflect 
the same pattern. 

7.3 Seek to shift the membership of the Board of Trustees in a 
similar direction. 

7.4 Review its present financial commitments and completely 
analyze available supplementary funds in the area of equal edu- 
cational opportunity programs, with a view toward a dramatic 
extension of activity. 

(Proposed by John C. Hoy in “Educational Civil Rights and the 
College Board”) 

8.1 Take steps to increase greatly the scope of its testing program 
to include the measurement of student competencies other than 
those reflected in measures of academic promise for college en- 
trance. 

8.2 Undertake initially a major study, including a pilot program 
to develop appropriate materials and procedures, of the feasibil- 
ity of assessing qualifications and classifications for job entry and 
for postsecondary technical training. 
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8.3 If the results of the study and the experience with the pilot 
program are promising, consider expanding its function to serve 
all high school graduates, including those entering the work 
force directly as well as those planning to go to college, and 
changing its name accordingly to “Career Entry. Examination 
Program,” or such other appropriate name as would accurately 
reflect its expanded clientele. 

(Proposed by Sidney P. Marland in “A Proposal for a Compre- 
hensive System of Testing for Job Entry”) 

9.1 Support a research and development program to assemble, 
evaluate, and interpret information about colleges, taking into 
account the kinds of issues, sources, and technical matters de- 
scribed in this brief. 

9.2 Regard this program as contributing, in stages, to the neces- 
sary software underlying a computerized information system. 

9.3 Present information developed from the program in some 
printed form to be determined by the College Board, pending 
its incorporation into a computerized system, so that as much as 
possible will be available to applicants at the earliest feasible time. 

(Proposed by C. Robert Pace in “A Program for Providing In- 
formation about Colleges to Applicants”) 

10. 1 Adopt as its principal policy objective in testing the provi- 
sion of a broad array of tests and other information designed to 
assist tenth- and eleventh-grade high school students in educa- 
tional planning and decisions with respect to post-high school 
opportunities. 

10.2 Provide factual and scientifically based information about 
individual colleges and universities for use by prospective appli- 
cants. This information should go beyond that now routinely 
supplied by institutions of higher education and should include 
ability and performance expectations for admission and for suc- 
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cess in particular programs as well as information about the cam- 
pus environment. 

10.3 Make provisions for a scries of three to five specialized re- 
porting programs, corresponding to different types of institu- 
tions of higher education, for the provision of test results to col- 
leges and universities in connection with institutional decisions 
about admissions and placement. Further, experimentation 
should be undertaken with respect to computer storage and re- 
trieval of this information so that it can be made available to the 
institutions at the time and in the form needed for decisions. 
Further, standards for protecting the privacy of students should 
be established so that any test information necessary for students 
but not appropriate for use in institutional decisions will be re- 
ported only to students. 

10.4 Be prepared to offer supplementary testing programs for 
particular groups of colleges and universities and for prospective 
applicants who may not be enrolled in high school under provi- 
sions that would limit redundancy and duplication with the basic 
program. 

10.5 Keep the objectives proposed in this report under continu- 
ing policy review and provide adequate empirical research in 
continuing review in order to assess the extent and timing of the 
proposed changes. 

(Proposed by Richard Pearson in “Education and Testing in a 
Period of Qualitative Change”) 

1 1. 1 Establish regional centers for guidance in continuing edu- 
cation. 

1 1.2 Consider as potential programs for these centers: 

a. Offering appraisal and advisement services; 

b. Developing a system of credit equivalency; 

c. Offering courses in guidance for credit; 

d. Using new training methods for staff development; 
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e. Conducting experimentation and research. 

1 1.3 Incorporate student participation into the regional centers 
and into the Board’s national program. 

1 1.4 Change its name and function to “Continuing Education 
Entrance Board.” 

(Proposed by Nancy K. Schlossberg, John C. Hoy, and Edmund 
W. Gordon in “Regional Centers for Guidance in Continuing 
Education”) 

12. 1 Supplement its present programs with self-administered and 
self-scored tests, made available to students in great profusion, 
and accompanied, under various degrees of delayed availability, 
by reasoned discussions about the choice of an answer for each 
item. 

12.2 Draw teachers, particularly those in secondary schools, into 
large-scale participation in the production both of these tests, 
and of tests in the conventional “adversary” mode. 

12.3 Stress face or curricular validity in the selection of test items; 
stress items of teaching merit. 

12.4 Invite, experimentally at least, groups of students who have 
taken a test soon thereafter to participate in an oral discuss/ n of 
the test under the Board’s auspices. 

(Proposed by B. Alden Thresher in “A Proposal for Self-Scored, 
Self-Administered Tests”) 

1 3. 1 Experiment actively with a system of modular, diagnostic 
instruments designed to test important competencies; these 
would, in principle, resemble the present achievement tests. 

13.2 Make these available to colleges as a means of permitting 
greater flexibility in the choice of combinations of tests selected 
by the student, and required or permitted by the college. 
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13.3 Continue the Scholastic Aptitude Test for the time being, 
with such improvements as inay be possible. 

13.4 Encourage member colleges gradually to replace the Schol- 
astic Aptitude Test with its fixed, predetermined parts or mod- 
ules, by more flexible assortments of modular tests. 

13.5 Reduce the spccdcdncss of its tests by reducing the num- 
ber of items per test, even at the cost of some loss of reliability. 

(Proposed by B. Alden Thresher in “Diversification in Educa- 
tional Assessment’’) 

14. 1 Establish a small study group that will outline an admissions 
machine and undertake feasibility studies needed to reach within 
two years: 

a. A better-grounded decision on the further possibility of 
developing an admissions machine; and, 

b. If then still considered possible, a more detailed financial 
and structural plan for implementing what appear to be better 
lines of its development. 

14.2 I11 appointing this study group, recognize that its report 
within two years will be:;r serious financial implications for the 
Board. 

(Proposed by David V. Tiedeman in “Can a Machine Admit an 
Applicant to Continuing Education?”) 
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